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Chapter  1 


INTRODUCTION 

Uncertainty  is  present  in  all  Major  Weapon  System 
Acquisitions.  Design,  performance,  maintainability,  produc¬ 
tion,  schedule,  and  cost  all  present  areas  where  substantial 
risk  is  present  for  the  Department  of  Defense  in  all  weapon 
acquisitions.  As  a  weapon  system  travel  through  the  acqui¬ 
sition  cycle,  it  reaches  the  point  where  full  scale  develop¬ 
ment  has  been  completed  and  production  is  ready  to  begin. 

This  transition  point  is  a  milestone  within  the  Major  System 
Acquisition  Cycle  and  is  aptly  named  Decision  Milestone  III 
(15:16-19).  At  this  point  in  the  acquisition  cycle,  the 
question  arises  of  whether  or  not  the  contractor  is  prepared 
to  begin  production  of  the  equipment.  The  Production  Readi¬ 
ness  Review  (PRR)  is  the  technique  used  by  the  Department 
of  Defense  to  reduce  the  uncertainty  encountered  when  a  major 
system  faces  the  production  point.  Department  of  Defense 
Instruction  5000.38,  Production  Readiness  Reviews  (25:1), 
states  that  the 

.  .  .  objective  of  a  PRR  is  to  verify  that  the  pro¬ 
duction  design,  planning,  and  associated  preparations 
for  a  system  have  progressed  to  the  point  where  a  pro¬ 
duction  commitment  can  be  made  without  incurring  unac¬ 
ceptable  risks  of  breaking  thresholds  of  schedule,  per¬ 
formance,  cost,  or  other  established  criteria. 


It  is  the  policy  of  the  Department  of  Defense  to 
require  a  PRR  before  any  major  system  enters  the  production 
phase.  The  DOD  component  is  responsible  for  performing  the 
PRR,  then  reporting  findings  and  an  assessment  of  the  con¬ 
tractor's  production  readiness  to  the  Defense  System  Acqui¬ 
sition  Review  Council  (DSARC)  at  Decision  Milestone  III 
(25:1-2).  Air  Force  Systems  Command  requires  a  PRR  be  per¬ 
formed  by  the  progreun  office  with  responsibility  for  the 
system's  development  and  acquisition.  According  to  AFSCP 
84-4  (19:20): 

.  .  .  the  purpose  of  conducting  a  PRR  prograun  is  to 
identify  manufacturing  risks  by  determining  whether 
(1)  a  system  of  equipment  under  development  is  capaOsle  of 
being  economically  produced  at  required  rates;  (2)  all 
importamt  production  engineering  problems  encountered 
during  development  have  been  resolved;  (3)  the  contrac¬ 
tor  has  accomplished  adequate  planning  through  the 
production  phase. 

Within  the  Air  Force,  the  System  Prograun  Office  (SPO) 
is  the  Systems  Command  organization  responsible  for  the 
Major  System  Acquisition.  Thus,  it  is  the  agency  which  per¬ 
forms  the  PRRs  and  presents  the  findings  at  Decision  Mile¬ 
stone  III.  Through  the  PRR,  the  goal  is  to  find  the  con¬ 
tractor  in  a  state  of  production  readiness  (22:6). 

Security  Assistance  Environment 

The  PRR  is  indeed  applicaUsle  to  a  weapons  system 
acquisition  Involving  U.S.  contractors.  In  an  increasing  num¬ 
ber  of  cases,  PRRs  will  be  performed  on  foreign  firms  in  mul¬ 
tinational  programs  as  well.  An  overview  of  U.S.  Security 
Assistance  is  necessary  to  illustrate  the  increasing 
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cooperation  between  the  U.S.  government  and  foreign  firms. 

A  key  element  of  the  United  States*  implementation  of  foreign 
and  national  security  policy  is  through  the  transfer  of  U.S. 
defense  articles,  services,  training,  and  economic  assistance 
to  foreign  allies.  Today,  this  in^lementation  of  security 
policy  is  more  commonly  known  as  security  assistance. 

Security  assistance  can  be  traced  to  the  earliest  recorded 
military  history  euid  has  matured  through  time  to  what  is 
presently  defined  as  a: 

Group  of  progreuns  authorized  by  the  Foreign  Assis¬ 
tance  Act  of  1969,  as  amended,  and  the  Arms  Export 
Control  Act  of  1976,  as  ^Ullended,  or  other  related 
statutes  by  which  the  United  States  provides  defense 
articles,  military  training,  and  other  defense  related 
services,  by  grant,  credit,  or  cash  sales  in  further¬ 
ance  of  national  policies  amd  objectives  [23:306-307]. 

Since  the  1960s,  the  greatest  portion  of  U.S.  security 

assistance  to  our  allies  has  been  rendered  in  the  form  of 

military  hardware  amd  training  via  Foreign  Military  Sales 

(FMS) .  "FMS  is  a  type  of  security  assistance  where  the 

recipient  country  provides  reimbursement  for  the  defense 

aurticles  auid  services  [18:p.9-10] . "  The  doctrine  suggests 

that  supplying  this  hardware  and  training  to  our  allies  will 

allow  these  nations  to  acquire  the  capability  to  defend 

themselves.  It  further  suggests  this  will  ensure  peace  and 

thus  promote  U.S.  foreign  policy  amd  national  security 

interests.  This  strengthening  of  our  allies  is  vital  to 

the  U.S.  Total  Force  Concept  (10:78).  In  reciprocal  fashion, 

many  foreign  nations  rely  upon  the  U.S.  to  provide 


military  hardware  and  services  for  their  purchase. 

The  security  of  our  friends  and  allies  contributes 
directly  to  the  secxirity  of  the  United  States.  For  over 
thirty  years,  the  U.S.  has  made  available  material  ser¬ 
vices,  and  training  to  friendly  countries  to  enedsle  them 
to  improve  their  own  defense  capabilities  [26:223]. 

Thus  the  sale  of  military  equipment  to  foreign  countries  is 

of  material  benefit  to  both  the  United  States  and  the  foreign 

customer.  This  trend  can  be  expected  to  continue  in  the 

future  (10:78-82). 

As  the  coR^lexity  and  sophistication  of  the  weapons 
system  increases  so  too  does  the  price  of  the  new  hardware. 
For  many  of  our  allies,  this  results  in  a  twofold  problem. 
First,  due  to  the  high  cost  of  these  new  weapons  systems, 
the  purchase  price  of  the  system  may  approach  50%  or  more 
of  that  country's  total  annual  budget.  Second,  the  direct 
purchase  of  weapons  systems  from  the  U.S.  would  limit  the 
purchasing  country's  abilities  to  maintain  its  own  industrial 
base  which  could  hurt  employment  and  balemce  of  payments. 

The  following  quotation  represents  the  feeling  of  many  of 
our  allies: 

The  United  States'  formerly  dominant  position  as 
the  principle  supplier  of  NATO's  defense  equipment  has 
been  increasingly  eroded.  Straight  sales  of  U.S. 
weapon  systems  to  European  NATO  nations  are  now  diffi¬ 
cult  or  impossible  to  achieve  [7:3]. 

Our  allies  in  this  situation  find  such  expensive  weapon 

systems  more  attractive  when  the  selling  country  offers  some 

percentage  of  the  production  and  development  work  to  the 

purchasing  country.  Two  alternatives  are  available  to  the 


purchasing  country:  (1)  secure  a  license  to  produce  the 
system  themselves,  or  (2)  enter  into  a  coproductive  arrange¬ 
ment  (18;pp. 13. 1-13. 2) . 

Coproduction  encompasses  any  prograun  wherein  the 
U.S.  Government,  under  the  aegis  of  an  international 
diplomatic  level  or  Ministry  of  Defense  to  Department 
of  Defense  agreement,  either  directly  through  the  FMS 
progrcun,  or  indirectly  through  specific  licensing 
arrangements  by  designated  commercial  firms,  enables  an 
eligible  foreign  government, . international  organization 
or  designated  commercial  producer  to  acquire  the  "know 
how"  to  manufacture  or  assemble,  repair,  maintain  and 
operate,  in  whole  or  in  part,  a  specific  weapon, 
communication  or  support  system,  or  individual  military 
item  [24:2] . 

This  situation  has  led  to  wide  discussion  of  the 
"two  way  street"  in  arms  transactions  within  NATO.  In  the 
Alliance  arms  sales,  the  phrase  "two  way  street"  is  used 
to  indicate  the  Europeans'  desire  for  mutual  sales  and 
coproduction  (7:3,16).  United  States  allies  have  a  strong 
desire  to  share  in  coproduction  of  major  systems  and  to 
share  in  revenue  gained  by  sales  to  third  countries. 

The  1979  GAO  report,  A  New  Approach  is  Needed  for 
Weapons  System  Coproduction  Progreuns  Between  the  United 
States  and  Its  Allies  (28:3),  states  that  the  coproduction 
trend  will  continue  in  the  future. 

Coproduction  will  likely  be  a  frequent  method  of 
weapons  system  acquisition  in  the  future,  according  to 
discussions  with  U.S.  and  foreign  government  and 
industry  officials.  Some  of  our  allies  will  continue 
to  require  costly  and  sophisticated  weapons  systems 
which  they  would  have  difficulty  producing  on  their  own. 
Others  may  wish  to  codevelop  systems  in  order  to  share 
the  cost  and  be  in  a  position  to  subsequently  coproduce 
them. 
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It  is  generally  felt  that  coproduction  progreuns  will  be  the 
rule  in  the  future  as  the  costs  of  military  hardware  con¬ 
tinue  to  increase. 

Coproduction  arrangements  will  allow  the  production 
of  specific  equipment  and  components  in  several  countries. 

In  many  cases,  the  foreign  contractor  may  be  producing  not 
only  for  foreign  consumption  but  for  United  States  consump¬ 
tion  as  well.  As  millions  of  dollars  worth  of  contracts 
and  subcontracts  are  awaurded  to  foreign  firms,  the  United 
States  faces  a  marketplace  filled  with  risk.  The  uncer¬ 
tainty  present  in  all  weapons  system  acquisitions  is  increased 
when  dealing  in  the  foreign  marketplace  because  of  the  many 
instances  where  foreign  development  and  manufacturing  tech¬ 
nology  is  behind  that  of  the  United  States.  When  foreign 
participants  actually  build  a  major  segment  of  the  equipment 
they  purchase  and  produce  parts  of  a  system  the  United  States 
will  use,  the  Production  Readiness  Review  (PRR)  becomes  a 
critical  measure  of  the  foreign  firm's  fitness  for  the  pro¬ 
duction  phase  (8:259;  28:5). 

The  F-16  Multinational  Coproduction  Progreun  has  set 
a  precedent  in  international  industrial  defense  cooperation. 
Referred  to  as  the  "Grand  Experiment,"  it  has  been  an 
attempt  to  both  partially  fulfill  economic  needs  of  the 
Western  Alliance  amd  satisfy  the  coiiq>anion  military  defense 
requirements  {13:p.l-l).  The  coproduction  of  the  F-16  by 
four  European  Participating  Governments  (EPGs) — Belgium, 
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Netherlands,  Norway,  and  Denmark — is  the  largest  multinational 
coproduction  program  attempted  to  date  (6:580-H).  Over  two 
billion  dollars  in  contracts  have  been  awarded  to  partici¬ 
pating  European  manufacturers.  Segments  are  being  produced 
for  all  sections  of  the  aircraft:  air  frame,  engine,  elec¬ 
tronics,  radar,  and  landing  gear.  In  many  cases,  parts 
produced  by  the  foreign  firms  will  be  used  on  U.S.  aircraft 
(16:59).  So  far,  the  F-16  program  has  demonstrated  that 
multinational  collaborative  efforts  can  be  implemented,  and 
that  "the  F-16  will  ultimately  be  considered  an  economic 
as  well  as  military  success  by  most  (if  not  all)  of  the 
nations  involved  [13;p.l-l]." 

Production  Readiness  Reviews  were  utilized  by  the 
F-16  System  Program  Office  (SPO)  to  ensure  that  foreign 
companies  involved  were  prepared  to  enter  the  production 
phase  of  the  program.  The  PRRs  performed  have  received  high 
marks  from  both  OSARC  (14)  amd  from  participants  interviewed 
by  Maxfield  Associates,  the  research  institute  publishing 
the  F-16  lessons  learned  report. 

Generally,  the  PRR's  have  been  planned  and  executed 
without  significant  difficulty.  The  deficiencies  and 
potential  problem  areas  identified  during  PRR's  have, 
as  a  rule,  been  expeditiously  corrected  with  the  coopera¬ 
tion  of  the  elements  of  European  and  U.S.  industry 
affected.  It  is  generally  felt  that  PRR's  contributed 
to  industry  auid  government  confidence  in  the  manufactur¬ 
ing  capability  of  the  European  Industrial  Elements 
involved  [13 :p. 3-ii-16] . 

The  F-16  PRR  program  reviewed  foreign  contractors  in 
four  European  coiintries — Belgium,  Denmark,  Netherlands,  and 
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Norway.  Dealing  with  foreign  firms  presented  new  industrial 
management  challenges  and  increased  the  uncertainty  involved 
in  the  overall  weapons  system  acquisition.  In  a  coproductive 
program  like  the  F-16  effort,  the  product's  cost,  quality, 
and  schedule  are  greatly  affected  by  each  of  the  partici¬ 
pating  foreign  contractors.  A  delay  in  the  production  of 
one  component  by  a  single  foreign  manufacturer  could  affect 
the  delivery  schedule  of  the  overall  system.  The  PRRs  per¬ 
formed  in  the  F-16  progreun  were  awarded  high  marks  for  their 
content  and  contributions.  The  late-Multinational  Manager 
at  the  F-16  SPO  (14)  indicated,  however,  that  many  variances 
existed  between  the  methodology  of  individual  foreign  PRRs. 
Although  loosely  based  on  the  1971  AFSCR  84-2  (20) ,  the 
reviews  did  not  follow  any  specific  guidelines.  Different 
survey  team  chiefs  performed  reviews  designed  and  planned 
largely  by  themselves.  These  reviews  placed  emphasis  on 
certain  areas  they  felt  were  most  important  relative  to 
production  readiness  of  the  foreign  coproducer. 

Problem  Statement 

The  late-Multinational  Manager  at  the  F-16  SPO  felt 
the  areas  of  emphasis  in  the  foreign  PRR  must  be  clarified. 
The  uQ>ortant  areas  for  manufacturing  and  production  review 
should  be  identified  and  then  compiled  into  a  set  of  areas 
requiring  special  emphasis  during  a  foreign  PRR. 
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Researrch  Objectives 

The  following  research  objectives  have  evolved  from 
the  problem  statement  and  were  pursued  in  this  study. 

1.  Determine  if  similar  areas  of  emphasis  exist 
among  different  foreign  PRR  teams. 

2.  Determine  if  differences  exist  between  areas  of 
emphasis  in  domestic  amd  foreign  PRRs. 

Research  Hypotheses 

Hypothesis  I:  Different  PRR  teams  concentrated  on 
similar  areas  of  emphasis  when  performing  PRRs  on  foreign 
firms  in  the  F-16  program. 

Hypothesis  II;  PRRs  performed  on  foreign  firms  have 
different  areas  of  emphasis  than  PRRs  performed  on  domestic 
firms. 


PRRs  are  used  by  the  Department  of  Defense  to  reduce 
the  manufacturing  risk  during  a  major  system  acquisition. 

Not  only  are  domestic  contractors  involved  in  the  U.S. 
weapons  acquisition,  but  today  foreign  contractors  are 
involved  as  well.  The  research  objectives  and  matching 
hypotheses  have  been  developed  to  study  the  applicedjility  of 
the  PRR  process  to  multinational  coproduction  progr2uns.  The 
following  chapter  outlines  background  information  and  data 
available  on  this  subject. 


chapter  2 


BACKGROUND 

The  1976  AFIT  Thesis  by  Brechtel  and  Lathrop 
addressed  the  applicability  of  Production  Readiness  Reviews 
to  DOD  acquisitions  from  United  States’  industry.  Their 
research  involved  a  review  of  the  guidelines  and  procedures 
for  performing  PRRs  and  then  reviewed  the  PRR  progreuns  for 
three  different  major  systems,  the  A“10,  F-15,  and  AWACS. 

The  three  major  systems  had  completed  the  entire  PRR  process 
beginning  with  the  planning  stage  and  culminating  with  the 
final  DSARC  review  at  Decision  Milestone  III.  A  comparison 
between  the  three  programs  allowed  Brechtel  and  Lathrop  to 
make  two  findings:  (1)  PRR  approaches  used  to  date  have 
been  different,  and  significant  disparities  did  exist  among 
the  three  PRR  programs  reviewed  by  the  thesis;  (2)  standard¬ 
ization  could  be  obtained  in  future  PRR  programs  if  AFSC 
exp>erts  could  agree  on  what  areas  should  be  emphasized  and 
how  best  to  conduct  future  PRR  progrcuns  (4:2-4,33,82-84). 

Brechtel  and  Lathrop  touched  only  major  systems 
where  United  States  contractors  were  involved.  There  are 
many  facets  of  the  international  marketplace  that  are  con¬ 
siderably  different  than  the  domestic  marketplace 
(14;  13:p.l-2).  There  are  numerous  instances  where 


manufacturing  technology  is  behind  that  of  the  United 
States  (8:259).  For  many  European  industries,  the  issue  of 
domestic  employment  is  fundamental  in  all  future  industrial 
planning.  In  many  cases,  stability  in  employment  is  more 
important  to  the  firm  than  profits  (7:21).  The  F-16  SPO 
teeuns  found  less  emphasis  on  quality  control  efforts  within 
foreign  contractors'  plants  than  within  domestic  contractors’ 
plants.  The  foreign  worlc  ethic  is  different  than  the  United 
States  ethic.  Production  in  most  foreign  industries  slows 
to  a  near  stop  during  July  when  roost  employees  ta)ce  their 
vacation.  National,  religious,  and  local  holidays  frequently 
halt  the  production  line,  in  Denmarlc  and  Belgium,  one  or  two 
day  wildcat  strilces  are  common  when  the  weather  is  alluring. 
For  U.S.  industries,  such  production  irregularities  would 
have  devastating  results.  Thus,  the  attitude  and  mood  of 
European  industry  is  somewhat  different  than  that  of  feuniliar 
U.S.  industry  (14). 

The  large  number  of  taxpayer  dollars  allocated  to 
the  production  phase  of  major  weapons  system  contracts  justi¬ 
fies  research  in  the  area  of  Production  Readiness  Reviews. 

The  F-16  multinational  coproduction  program  has  awarded  over 
two  billion  dollars  in  contracts  and  subcontracts  to  foreign 
industry  (16:59).  The  technical  uncertainty  in  doing  foreign 
business  creates  risks  that  can  be  reduced  by  effective  use 
of  Production  Readiness  Reviews.  The  Brechtel  and  Lathrop 
thesis  identified  replication  of  the  thesis  as  an  area  for 
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future  research  (4:101-102).  The  importance  of  foreign 
industry  to  future  coproduction  efforts  makes  research  in 
the  area  of  foreign  production  readiness  reviews  a  worth¬ 
while  endeavor. 

Related  Research 

The  background  review  was  seeded  by  the  1976  AFIT 
thesis  by  Brechtel  and  Lathrop.  The  majority  of  research 
efforts  on  Production  Readiness  Reviews  up  to  1976  were  pri¬ 
marily  government  initiated  (4:29).  The  research  studies 
concentrated  on  information  gathering  techniques  to  cope  with 
the  high  uncertainty  present  when  entering  the  Production 
Phase  of  Major  System  Acquisitions.  Research  performed 
since  1976  Ceui  be  categorized  into  three  basic  areas: 

(1)  general  research  on  Production  Readiness  indicating  the 
need  for  some  evaluation  of  a  contractor's  ability  to  proceed 
to  production,  (2)  expert  testimony  by  AFSC  manufacturing 
personnel  involved  in  the  performance  of  PRRs,  and  (3)  scien¬ 
tific  research  on  how  to  apply  the  PRR  techniques. 

Brechtel  and  Lathrop 's  Study 

Brechtel  amd  Lathrop 's  1976  thesis,  A  Comparative 
Analysis  of  the  Application  of  Production  Readiness  Reviews 
(4) ,  studied  three  major  weapons  system  progreuns  that  had 
completed  an  entire  PRR  program  and  had  met  DSARC  III  review. 
Included  in  the  population  of  interest  were:  (1)  Airborne 
Warning  and  Control  System  (AWACS) ;  (2)  F-15  Air  Superiority 


Fighter;  and  (3)  A-10  Close  Air  Support  Attack  aircraft. 

All  three  systems  underwent  PRR  programs  that  began  with  the 
planning  stage  and  culminated  in  DSARC  III  approval.  At 
that  time,  AFSCR  84-2,  Production  Readiness  Review  (20), 
guided  members  of  the  System  Progreun  Office  as  they  planned 
their  PRRs. 

The  researchers  used  the  25  questions  from  the  1971 
regulation,  however,  to  compare  the  opinions  of  foreign  PRR 
experts  to  the  domestic  PRR  experts  of  Brechtel  and  Lathrop's 
study.  This  was  necessary  to  ensure  validity  of  the  com¬ 
parison  between  the  two  populations.  The  substance  of  the 
questions  has  not  changed;  only  the  arrangement  is  different. 
Thus,  the  researchers  feel  the  use  of  the  25  questions  from 
the  1971  regulation  is  valid  for  comparison  purposes. 

An  objective  of  Brechtel  and  Lathrop  was  to  deter¬ 
mine  if  a  standard  PRR  approach  was  feasible  to  satisfy 
AFSCR  €4-2  requirements.  They  approached  this  objective 
through  interviews  with  ten  production  management  experts 
within  Air  Force  Systems  Command  (AFSC) .  While  based  on 
three  criteria,  the  key  determinant  for  selecting  production 
management  experts  was  their  experience  in  previous  PRRs. 
During  interviews,  the  ten  experts  were  requested  to  rank 
the  25  standard  PRR  questions  listed  in  AFSCR  84-2  (20:5-6) 
in  order  of  importance  for  future  PRR  programs.  The  null 
hypothesis  (Hg)  was  that  no  difference  would  exist  among  the 
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mean  ranks  for  each  of  the  25  questions;  thus,  the  experts 
did  not  agree  on  the  importance  of  some  areas  over  others. 
The  alternate  hypothesis  (Hj^)  was  that  differences  did 
exist  among  the  means,  and  experts  did  agree  on  the  relative 
importance  of  some  areas  over  others.  The  Friedman  Two-Way 
Analysis-of-Variance-by-Ranks  test  (17:166)  was  used  to  test 
the  research  hypothesis. 

Domestic  PRR  findings.  Brechtel  and  Lathrop  found 
that  the  ten  AFSC  production  management  experts  did  agree 
on  the  relative  importance  of  all  25  AFSCR  84-2  questions 
(4:70-71).  The  mean  rank  was  coit5>uted  for  each  question 
and  the  questions  then  assembled  in  order  of  their  relative 
mean  rank  (4:144-147).  The  experts  generally  agreed  that 
the  areas  at  the  top  of  the  list  should  receive  the  greatest 
emphasis,  and  the  areas  at  the  end  of  the  list  should 
receive  the  least  emphasis.  Additionally,  Brechtel  and 
Lathrop  noted  that  six  of  the  25  areas  had  mean  values  much 
lower  (highest  ranking)  than  the  other  19  areas.  Of  these 
six  areas,  five  addressed  the  completion  of  engineering  and 
testing  in  some  manner,  thus  there  was  high  importance 
placed  on  the  completion  of  engineering  tasks  prior  to  the 
start  of  the  production  phase.  Finally,  Brechtel  and 
Lathrop  noted  that  the  order  in  which  the  questions  were 
presented  in  AFSCR  84-2  did  not  agree  with  the  prioritized 
order  presented  by  the  ten  production  management  experts. 
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Thus,  they  felt  the  1971  AFSCR  84-2  had  not  prioritized  the 
25  questions  in  a  manner  useful  to  the  System  Program  Office 
charged  with  the  responsibility  of  conducting  a  Production 
Readiness  Review  (4:82-85).  This  has  been  addressed  by  the 
1981  rewrite  of  AFSCR  84-2. 

General  research.  Several  unpublished  papers  have 
been  located  which  concern  production  readiness.  Primarily, 
the  papers  deal  with  the  question  of  what  areas  should  be 
reviewed  in  a  PRR  (1;  6;  12).  Additionally,  the  usefulness 
of  a  PRR  in  assessing  production  readiness  was  discussed. 

General  research  revealed  that  the  Air  Force  has 
been  performing  PRRs  for  over  ten  years.  Air  Force  Systems 
Command  Regulation  84-2,  first  pxiblished  in  November  1971, 
has  been  the  forerunner  of  most  DOD  regulations  concerning 
PRRs.  In  January  1979,  the  DOD  published  DOD  5000.38, 
Production  Readiness  Reviews  (25) .  Following  the  issuance 
of  this  DOD  instruction,  each  service  component  published 
its  own  guidance  for  performing  PRRs.  Additionally,  Air 
Force  Systems  Command  has  rewritten  AFSCR  84-2  and  recently 
published  a  1981  version  of  the  regulation  (21) . 

In  addition  to  existing  regulations,  a  community  of 
PRR  interest  is  growing  within  the  DOD.  This  is  evidenced 
by  the  November  1980  DOD/Joint  Services  Production  Readiness 
Reviews  Conference  held  at  Wright-Patterson  APB.  The  objec¬ 
tive  of  the  conference  was  to  provide  a  forum  for  people 
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involved  in  the  PRR  process  to  exchange  lessons  learned  and 
ideas  for  improving  the  PRR  process  {21) .  The  output  of  the 
conference  was  a  list  of  14  observations  and  action  items 
concerning  how  the  PRR  process  could  be  improved.  None  of 
these  observations  addressed  foreign  PRRs.  While  foreign 
PRRs  were  given  a  quick  brush  stroke  during  the  conference, 
no  substantive  discussions  were  held  concerning  the  unique 
challenges  of  the  foreign  PRR. 

Finally,  general  research  showed  DOD  Production 
Engineering  Services  Office  (DPESO)  performed  a  Cost/Benefit 
Study  of  Production  Readiness  Reviews  and  presented  it  as 
introductory  information  at  the  November  1980  DOD  Conference. 
The  report  noted  "PMOs  (Project  Mcuiagement  Offices)  conducting 
foreign  PRRs  have  problems  not  encountered  on  domestic 
PRRs  [27:12]."  Aside  from  this  brief  mention  in  the  DPESO 
Cost/Benefit  Study,  no  attention  has  been  focused  upon  the 
unique  problems  posed  by  performance  of  PRRs  on  foreign 
coproducers . 

Expert  testimony.  Within  the  F-16  SPO,  several 
participants  in  previous  F-16  PRR  progreuns  have  been  inter¬ 
viewed.  Information  provided  has  centered  primarily  on  the 
differences  between  domestic  and  foreign  contractors  and  the 
variances  between  PRRs  conducted  on  different  F-16  foreign 
contractors  (14) .  Additionally,  Major  Lyle  Lockwood,  former 
Deputy  Director  of  the  Air  Force  Business  Management  Research 


Center  and  long  an  observer  of  PRRs,  has  provided  his  exper¬ 
tise  on  the  Air  Force  PRR  system  (11). 

Lieutenant  Colonel  Thomas  Fiorino,  Director  of  Manu¬ 
facturing  auid  Quality  Assurance  for  the  F-16  SPO,  provided 
invaluable  insight  into  the  foreign  PRRs  of  the  F-16  pro¬ 
gram  (5) .  Further,  Major  Don  Brechtel  was  interviewed.  His 
detailed  knowledge  of  the  PRR  process  was  of  tremendous  value 
when  assembling  a  chronology  for  the  development  of  the 
PRR  (3).  Aside  from  the  actual  performance  of  a  PRR  on  a 
foreign  firm,  these  men  knew  of  no  attempts  to  study  the 
application  of  PRRs  to  foreign  contractors. 

Scientific  research.  Brechtel  and  Lathrop's  1976 
thesis  holds  most  of  the  scientific  research  conducted  on 
the  Production  Readiness  Review  to  date.  No  other  research 
was  located  that  dealt  solely  with  the  application  of  PRRs 
in  domestic  industry.  The  Defense  Logistics  Studies  Infor¬ 
mation  Exchange  and  the  Defense  Documentation  Center  have 
no  information  dealing  directly  to  PRRs. 

Summary 

In  no  case  was  any  material  located  dealing  with 
the  application  of  PRRs  to  foreign  contractors.  Only  the 
experience  of  manufacturing  experts  within  the  F-16  SPO 
provides  the  link  between  PRRs  and  the  multinational  copro¬ 
duction  progreun.  Thus,  the  replication  of  Brechtel  and 
Lathrop's  thesis  on  the  F-16  multinational  progreun  provides 
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fruitful  research  concerning  the  applicability  of  PRR  tech¬ 
niques  to  multinational  coproduction  programs.  The  follow¬ 
ing  chapter  discusses  the  comparison  of  domestic  and  foreign 
firms,  identifies  the  population  studied,  and  then  describes 
the  procedures  used  in  analysing  data  obtained  in  the 


research. 


Chapter  3 


RESEARCH  METHODOLOGY 

The  F-16  PRR  program  reviewed  the  contractors  in 
four  European  countries — Belgium,  Denmark,  Netherlands,  and 
Norway.  The  previous  two  chapters  discussed  the  importance 
of  these  reviews  and  highlighted  the  need  to  reduce  the 
uncertainty  involved  when  dealing  with  foreign  contractors. 
Using  techniques  developed  by  Brechtel  and  Lathrop  in  their 
1976  thesis,  the  researchers  studied  participants  in  the 
F-16  program  and  sought  to  determine  if  the  participants 
share  common  areas  of  emphasis  when  performing  PRRs  on 
foreign  industrial  contractors.  This  chapter  discusses  the 
comparison  of  domestic  and  foreign  firms,  identifies  the 
population  studied,  and  then  describes  the  procedures  used 
in  analyzing  data  obtained  in  the  research. 

Comparison  of  Domestic  and 
Foreign  Firms 

The  three  major  weapons  system  programs  reviewed  by 
Brechtel  and  Lathrop  were  systems  developed  and  produced  by 
United  States  contractors.  Production  Readiness  Reviews  were 
performed  on  many  of  the  domestic  firms  involved,  and  the 
production  management  experts  interviewed  by  Brechtel  and 
Lathrop  were  experienced  in  the  area  of  domestic  PRRs. 


Thus,  the  prioritiration  of  the  25  AFSCR  84-2  questions  was 
a  ranking  of  the  areas  of  emphasis  for  domestic  PRR  programs. 
These  25  areas  have  been  listed  in  order  of  importance  for 
a  Production  Readiness  Review  perfoirmed  on  a  United  States 
contractor  (4:144-147). 

The  P-16  progr2un  has  introduced  foreign  contractors 
to  the  major  system  acquisition  cycle.  System  Program 
Offices  may  now  face  the  task  of  performing  PRRs  on  foreign 
firms  as  well  as  domestic  firms.  The  System  Program  Office 
for  the  F-16  has  performed  Production  Readiness  Reviews  on 
23  foreign  contractors.  The  procedures  used  by  Brechtel  and 
Lathrop  were  replicated  using  personnel  from  the  F-16  progreun 
to  study  the  areas  of  emphasis  in  foreign  PRRs.  The  objec¬ 
tives  of  this  study  were:  (1)  determine  similarities  that 
exist  between  areas  of  emphasis  for  different  foreign  PRR 
teams  and  (2)  determine  if  differences  exist  between  areas 
of  emphasis  for  domestic  PRRs  and  foreign  PRRs. 


Population  of  Interest 

The  population  of  interest  consisted  of  individuals 
who  have  participated  in  PRRs  of  foreign  contractors  in  the 
F-16  program.  A  seunple  size  of  ten  was  used  by  Brechtel  and 
Lathrop.  The  seune  sample  size  was  used  in  this  study.  The 
ten  individuals  in  the  sample  have  been  identified  in 
Appendix  F.  The  Manufacturing  and  Quality  Assurance 


Directorate  of  the  F-16  System  Prograun  Office  is  the  office 
charged  with  responsibility  for  conducting  F-16  Production 
Readiness  Reviews. 


Research  Hypothesis  I 

The  questionnaire  technique  was  used  as  a  means 
for  collecting  data  to  test  Research  Hypothesis  I.  A  sample 
of  the  questionnaire  has  been  furnished  in  Appendix  A. 
Personnel  within  the  F-16  SPO  who  had  prime  responsibility 
for  performing  PRRs  were  asked  to  complete  the  questionnaire 
and  were  asked  to  rank  the  25  standard  PRR  questions  listed 
in  the  1971  AFSCR  84-2  (20:5-6)  in  order  of  importance  for 
future  foreign  PRR  prograuns.  The  25  questions  have  been 
furnished  in  Appendix  B.  The  specific  instructions  used 
when  requesting  the  F-16  personnel  to  rank  the  questions 
contained  the  following  statements: 

This  part  of  the  questionnaire  aims  at  determining 
whether  a  standard  PRR  approach  can  be  developed  to 
practically  be  applied  to  various  foreign  contractors 
in  a  major  system  acquisition.  Additionally,  the 
questionnaire  will  determine  if  differences  exist  in  the 
areas  of  emphasis  of  PRRs  conducted  on  domestic  firms 
and  foreign  firms. 

Please  rank  the  25  PRR  questions  in  AFSCR  84-2  in 
order  of  importance  to  a  foreign  PRR.  A  "one"  should 
be  assigned  to  the  PRR  question  that  should  have  the 
greatest  amount  of  time,  effort,  and  resources  expended 
for  analysis  and  reporting,  and  a  "25"  should  be  assigned 
to  the  least  important  question.  Please  rank  the  ques¬ 
tions*  according  to  how  you  feel  about  performing  a  PRR 
on  a  foreign  contractor. 
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Design  to  Test  Research 
Hy^thesis  I 

To  test  the  research  hypothesis  that  certain  PRR 
areas  require  special  emphasis  when  dealing  with  foreign 
contractors,  the  Friedman  Two-Way  Analysis-of-Variance-by- 
Ranks  test  (17:166)  and  the  Kendall  Coefficient  of  Concor¬ 
dance  W  (17:229)  test  were  used.  The  null  hypothesis  (Hq) 
for  bo  til  the  Friedman  test  and  the  Kendall  test  was  that 
there  was  no  difference  among  the  mean  ranks  for  the  25 
AFSCR  84-2  questions;  therefore,  the  F-16  PRR  personnel  did 
not  agree  on  the  rankings  for  the  25  questions.  The  alter¬ 
nate  hypothesis  (H^^)  for  both  tests  was  that  there  were 
differences  between  the  mean  ranks  for  the  25  AFSCR  84-2 
questions;  therefore,  the  F-16  PRR  personnel  agreed  on  the 
relative  rankings  for  the  questions.  If  the  PRR  personnel 
agreed  on  the  relative  rankings  of  the  questions,  then  it 
may  be  concluded  that  certain  areas  require  more  emphasis 
than  others  when  performing  a  foreign  PRR.  Both  tests  are 
discussed  in  the  following  chapter. 

Research  Hypothesis  II 

The  questionnaire  technique  used  for  Hypothesis  I 
was  also  used  for  Hypothesis  II.  The  mean  rank  for  each 
question  was  determined  from  the  ten  individual  rankings  of 
the  25  questions  used  in  testing  Hypothesis  I.  The  mean 
rank  determined  for  each  question  in  this  study  (foreign 
PRRs)  was  then  compared  to  the  mean  rank  obtained  from  the 
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Brechtel  and  Lathrop  study  (domestic  PRRs) .  The  comparison 
has  yielded  insight  into  the  disparities  between  areas  of 
emphasis  in  the  foreign  and  domestic  environment. 

Design  to  Test  Research 
Hypothesis  II 

To  test  the  research  hypothesis  that  PRRs  performed 
on  foreign  firms  have  different  areas  of  emphasis  than  PRRs 
performed  on  domestic  firms,  the  correlation  between  the  mean 
ranks  of  the  foreign  PRR  experts  and  those  of  the  domestic 
PRR  experts  were  studied.  The  Spearman  Rank  Correlation 
Coefficient  (17:202;  9:275)  and  the  Kendall  Rank  Correlation 
(17:213;  9:284)  tests  were  used.  High  positive  correlation 
indicated  that  there  was  little  difference  in  the  areas  of 
emphasis  between  the  two  populations;  whereas  a  high  negative 
correlation  indicated  that  there  was  great  difference.  No 
correlation  indicated  neither  agreement  nor  disagreement  in 
the  areas  of  emphasis  in  the  two  populations.  The  null 
hypothesis  (Hq)  for  the  correlation  tests  was  that  there 
were  differences  between  the  areas  of  emphasis  in  foreign 
and  domestic  PRRs;  therefore,  the  ranks  assigned  by  the 
individuals  in  the  F-16  SPO  were  different  from  the  ranks 
assigned  by  the  individuals  studied  by  Brechtel  and  Lathrop. 
The  alternate  hypothesis  (H^)  was  that  there  was  no  differ¬ 
ence  between  the  areas  of  emphasis  in  foreign  and  domestic 
PRRs*  therefore,  the  ranks  assigned  to  the  25  questions  were 
the  same  for  both  foreign  and  domestic  participants.  The 
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mechanics  for  both  the  Spearman  test  and  the  Kendall  test 
have  been  discussed  in  detail  in  the  following  chapter. 

If  differences  were  determined  to  exist  between  the 
areas  of  emphasis  in  foreign  PRRs  and  domestic  PRRs,  then 
the  differences  could  be  identified  and  addressed  to  aid 
planning  and  execution  for  future  foreign  PRRs.  A  study 
ensued  to  determine  the  applicability  of  the  PRR  process  in 
multinational  coproduction  programs.  The  insight  of  the 
F-16  PRR  participants  have  been  helpful  in  approaching  this 
question. 


Additional  Research  Data 

Not  only  was  the  questionnaire  technique  used  as  a 
means  for  collecting  data  to  test  Research  Hypotheses  I  and 
II,  but  the  questionnaire  was  also  used  as  a  means  for 
collecting  subjective  data  from  the  foreign  PRR  experts  con¬ 
cerning  the  PRR  process. 

The  first  part  of  the  questionnaire  not  only  had  the 
foreign  PRR  experts  rank  the  25  questions  from  one  to  25, 
but  it  also  asked  them  to  prioritize  each  question  in  a 
ranking  from  A  to  E.  An  assignment  of  A  indicated  highest 
priority  of  importance,  while  an  assignment  of  E  indicated 
the  lowest  priority  of  importance.  This  data  was  used  to 
help  determine  the  importance  of  each  question  to  the 
foreign  PRR  expert.  The  data  obtained  from  this  test  compli¬ 
mented  the  test  for  Hypothesis  I,  and  the  mean  priorities 
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for  each  question  with  an  explanation  on  how  these  means 
were  computed  are  furnished  in  Appendix  A. 

The  questionnaire  also  contained  questions  which 
were  designed  to  gather  subjective  data  on  the  PRR  process 
from  the  foreign  PRR  experts-  These  questions  were  contained 
in  Part  II  of  the  questionnaire  (Appendix  A) .  The  questions 
pertained  to  the  timing  of  the  PRR  process,  to  the  interest 
shown  by  the  prime  and  associated  U.S.  contractors  on  the 
foreign  firms,  and  to  the  feelings  of  the  foreign  PRR  experts 
as  to  their  perceived  role  in  the  PRR  process.  The  final 
two  questions  asked  the  foreign  PRR  expert  to  identify  areas 
in  which  the  greatest  difference  existed  between  foreign  and 
domestic  PRRs  and  to  relate  a  memorable  personal  experience 
or  valuable  PRR  experience  while  conducting  a  PRR.  The 
cumulative  results  from  the  responses  to  appropriate  ques¬ 
tions  in  Part  II  of  the  questionnaire  have  been  furnished 
in  Appendix  A.  Selected  solicited  and  unsolicited  comments 
to  these  questions  in  Part  II  and  to  other  parts  of  the 
questionnaire  have  been  furnished  in  Appendices  H  and  I. 

By  collecting  and  analyzing  the  responses  of  the 
foreign  PRR  experts  to  the  questions  in  Part  II  of  the 
questionnaire,  valueUale  information  was  assembled  to  aid  in 
the  planning  euid  execution  of  future  foreign  PRRs. 
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Summary 

The  questionnaire  technique  was  used  as  the  data 
gathering  method  for  both  Hypothesis  I  and  Hypothesis  II. 
Using  the  results  of  the  Brechtel  and  Lathrop  thesis  and 
the  rankings  provided  by  F-16  personnel,  a  study  was  made 
of  the  differences  in  areas  of  emphasis  between  domestic  and 
foreign  PRRs.  In  addition,  the  foreign  PRR  experts  were 
asked  to  assign  one  of  five  priorities  to  each  of  the  1971 
AFSCR  84-2  questions.  The  experts  were  also  asked  to  give 
their  opinion  on  more  subjective  questions  which  were  used 
to  gain  a  better  understanding  of  the  PRR  process. 
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Chapter  4 


DATA  ANALYSIS 

Overview 

This  chapter  contains  the  specific  findings  emerging 
from  analysis  of  Research  Hypotheses  I  and  II,  and  corollary 
findings  brought  to  surface  by  the  questionnaire. 

The  data  indicates  that  PRRs  are  not  only  appliceible 
to  multinational  coproduction  programs,  they  are  imperative . 
Every  interview,  every  questionnaire,  and  every  consulted 
sovurce  indicated  that  the  foreign  PRRs  and  their  follow-ups 
contributed  to  the  overall  success  of  the  coproduction  pro¬ 
gram,  and  that  they  are  essential  to  good  management. 

Furthermore,  the  data  indicates  that  PRRs  performed 
at  foreign  subcontractor's  plants  presented  unique  challenges 
not  encountered  when  dealing  with  domestic  defense  contractors. 
The  data  revealed  many  areas  that  require  special  emphasis 
during  the  planning  and  execution  of  a  foreign  PRR. 

The  ten  PRR  "experts"  were  the  source  of  a  great 
deal  of  the  data  used  to  complete  this  study.  It  is  appro¬ 
priate  here  to  acknowledge  their  expertise  in  the  area  of 
Production  Readiness  Reviews  and  to  provide  a  brief  portrait 
of  their  experience.  The  ten  "experts”  consisted  of  the  six 
AF  personnel  and  four  civilian  consultants  listed  in 
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Appendix  F.  Collectively,  they  have  participated  in  over 
400  domestic  and  foreign  PRRs.  Of  the  average  40  PRRs  in 
which  each  man  has  participated,  an  average  of  26  were 
foreign  PRRs.  The  least  niamber  of  foreign  PRRs  performed  by 
any  respondent  was  six.  In  addition  to  the  ranking  of  the 
25  standard  PRR  questions,  each  man  provided  thoughtful 
responses  to  the  questionnaire  which  will  aid  in  the  planning, 
timing,  and  execution  of  future  foreign  PRRs. 

Research  Hypothesis  I 

Different  PRR  teauns  concentrated  on  similar  areas 
of  emphasis  when  performing  PRRs  on  foreign  firms  in 
the  F-16  program. 

Primary  Findings 

To  evaluate  Research  Hypothesis  I,  ten  PRR  experts 
who  had  participated  in  the  F-16  foreigi.  PRRs  were  asked  to 
respond  to  the  questionnaire  furnished  in  Appendix  A.  The 
personnel  were  asked  to  rcUik  the  25  standard  PRR  questions 
listed  in  AFSCR  84-2  (20:5-6)  in  order  of  importance  for 
future  PRR  programs.  If  the  F-16  PRR  personnel  agree  on  the 
relative  rankings  of  the  questions,  then  it  may  be  concluded 
that  the  different  PRR  teams  performing  F-16  foreign  PRRs 
concentrated  on  similar  areas,  and  that  the  25  standard 
questions  may  be  ranked  in  order  of  importance  to  a  foreign 
PRR. 
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First  Primary  Finding 

The  actual  ranking  data  from  the  ten  foreign  PRR 
experts  for  the  25  standard  PRR  questions  is  contained  in 
Appendix  E.  The  primary  nonparametric  statistical  test  for 
evaluating  the  agreement  among  the  ten  rankings  is  the 
Friedman  Two-Way  Analysis-of-Variance-by-Ranks  Test  (17:166). 
The  null  hypothesis  (Hq)  for  the  Friedman  test  was  that 
there  was  no  difference  among  the  mean  ranks  of  the  25 
AFSCR  84-2  questions;  thus,  the  F-16  personnel  do  not  agree 
on  rankings.  The  alternate  hypohtesis  (Hj^)  for  the  test 
was  that  differences  between  mean  ranks  did  exist;  thus, 
the  F-16  personnel  do  agree  on  the  relative  importance  of 
the  questions.  To  provide  additional  support  for  the 
Friedman  test,  the  nonparametric  Kendall  Coefficient  of 
Concordance  W  Test  (17:229)  was  performed  for  verification. 

As  discussed  in  the  following  section,  the  data  supported 
the  25  PRR  questions  may  be  ranked  in  order  of  impor¬ 
tance  for  a  foreign  PRR. 

Friedman  Two-Way  ANOVA  by  Ranks  test.  This  test  is 
used  on  data  from  K  matched  samples  which  are  in  an  ordinal 
scale,  and  is  useful  for  testing  to  determine  if  the  seunples 
have  been  drawn  from  the  same  population.  For  this  test 
the  data  were  placed  in  a  two-way  table  having  N  rows  and 
K  columns  (Appendix  E) .  The  rows  represent  the  ten  F-16 
production  management  experts,  and  the  columns  represent  the 


25  AFSCR  84-2  PRR  questions.  Each  row  gave  the  rank  score 


one  of  the  experts  gave  to  the  25  AFSCR  84-2  PRR  questions. 
The  scores  in  each  row  are  ranked  separately. 

The  procedure  used  in  conducting  the  Friedman  test 
was  covered  in  the  unpublished  master's  thesis  by  Brechtel 
and  Lathrop  (4:46-50),  and  from  the  text  on  Nonparametric 
Statistics  by  Sidney  Siegel  (17:166-173).  The  restatement 
of  the  null  hypohtesis  (H^)  and  the  alternate  hypothesis 
(Hj^)  is  in  order  at  this  time.  The  null  hypothesis  (Hq)  is 
that  there  is  no  difference  among  the  mean  ranks  for  the 
25  AFSCR  84-2  questions;  therefore,  the  F-16  PRR  personnel 
do  not  agree  on  the  rankings  for  the  25  questions.  The 
alternate  hypothesis  (H^^)  is  that  there  are  differences 
between  the  mean  ranks  for  the  25  AFSCR  84-2  questions; 
therefore,  the  F-16  PRR  personnel  agree  on  the  relative 
rankings  for  the  questions. 

The  formula  used  to  conpute  the  value  for  Ranks 
is  as  follows: 

x'  ■  Slc<k“  1)  / 

where 

N  >■  Number  of  rows 
k  >  Number  of  columns 
Rj  =  Sum  of  the  ranks  in  the  column 
k 

I  Directs  one  to  sum  the  squares  of  the  sums  of 
j>l  ranks  over  all  K  column 
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Therefore,  in  this  study: 

N  =  10 
k  =  25 

Rj  =  provided  for  in  Appendix  E  for  j  =  1,  2,...,  25. 
25 

E  (R^)^  =  453,095 
j=l 

So, 

Ranks  =  -(ioR25)  (26)  (^53,095)  -  (3)  (10)  (26) 

=  836.4831  -  780 
=  56.4831 

The  critical  value  for  the  Chi-square  distribution 
was  set  at  a  significance  level  of  .05  using  k- 1=25- 1*24 
degrees  of  freedom.  This  was  the  seune  pareuneter  used  in 
determining  the  critical  value  in  the  work  of  Brechtel  and 
Lathrop  (4:64) .  The  critical  value  was  obtained  from  the 
C..  square  distribution  table  (2:90)  and  determined  to  be 
36.4.  Since  the  Remks  *  56.4831  exceeded  the  critical 
value,  the  conclusion  was  made  to  reject  Hq  (the  null  hypoth¬ 
esis)  in  favor  of  (the  alternate  hypothesis) .  The  test 
concluded  that  the  F-16  SPO  PRR  experts  did  generally  agree 
on  the  prioritized  ranks  for  the  25  AFSCR  84-2  PRR  questions. 
This  was  also  the  s^une  conclusion  reached  in  the  Brechtel 
and  Lathrop  study  (4:64-65). 


Even  at  higher  signif iccmce  level  values,  the  con¬ 
clusion  was  still  the  same.  The  critical  value  at  the  .01 
level  of  significance  with  24  degrees  of  freedom  was  deter¬ 
mined  to  be  43.0,  amd  the  critical  value  at  the  .005  level 
of  significance  with  the  same  degrees  of  freedom  was  found 
to  be  45.6.  The  final  conclusion  from  the  Friedman  test  was 
that  the  ten  F-16  SPO  production  management  experts  did  agree 
on  the  prioritized  ranks  for  the  25  AFSCR  84-2  questions. 

To  verify  the  results  obtained  from  the  nonparametric 
Friedman  test,  one  additional  statistical  test  was  performed- 
the  nonparametric  Kendall  Coefficient  of  Concordance  W. 

Kendall  Coefficient  of  Concordance  W  (17;229-239) . 
This  test  is  useful  in  determining  the  association  among  the 
25  sets  of  rankings.  The  Kendall  test  expresses  the  degree 
of  association  ^unong  the  25  questions  and  is  particularly 
useful  in  studies  of  inter judge  or  interest  reliability  with 
applications  in  studies  of  clusters  of  variables.  The  pro¬ 
cedure  used  in  conducting  the  nonpareunetric  Kendall  Coeffi¬ 
cient  of  Concordance  W  test  is  summarized  in  Siegel  (17:237). 
The  null  hypothesis  (Hq)  for  this  test  is  that  the  F-16  SPO 
PRR  personnel ' s  rankings  of  the  questions  are  unrelated  to 
each  other.  The  information  provided  in  Appendix  E  was  used 
to  make  the  Kendall  Coefficient  of  Concordance  W  calculations 
The  mean  of  values  (Rj)  was  computed  as  follows: 
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where 


N  =  Number  of  entities  to  be  ranked  =  25 
Rj  =  Sum  of  rauiks  assigned  to  each  entity. 
Therefore , 


Rj  =  =  129.8 


It  was  then  required  to  find  the  sum  of  the  squared  devia¬ 
tions  (S)  for  all  25  R^  values  from  ^  which  was  calculated 
as  shown  below: 


S  =  Z{R^  -  rT)2 

=  31,894 


This  information  was  then  used  to  confute  the  coefficient 


W 


T5 


k^ (N®  -  N) 


where 

k  =  Number  of  sets  of  rankings  »  10. 
Therefore , 


W 


31,894 

^(100)  (15600) 


-  .24533846 


since  N  was  larger  than  seven,  the  following  formula  was 
used  to  calculate  a  Chi-square  value: 


=  - = -  =  k(N  -  1)W 

1  kN(N’  -  N) 

=  10(24) (.24533846) 

=  58.88123 

Again,  using  the  critical  value  obtained  from  the  Chi-squared 
distribution  table  of  36.4,  it  can  be  safely  concluded  to 
reject  Hq  (the  null  hypothesis)  at  a  level  of  significance 
of  .005  and  degrees  of  freedom  equal  to  N-l=24.  It  can  be 
concluded  with  considerable  assurance  that  the  agreement 
eunong  the  ten  P-16  SPO  production  management  experts  was 
higher  than  it  would  be  by  chance.  The  Kendall  Coefficient 
of  Concordance  W  results  gave  credence  to  the  outcome 
obtained  from  the  Friedman  Two-Way  Analysis-of-Variance-by- 
Ranks  test. 

AFSCR  84-2  PRR  question  priority  list.  Since  the 
ten  foreign  PRR  experts  did  agree  on  the  relative  importance 
of  all  25  AFSCR  84-2  questions,  the  means  of  each  of  the  ten 
rankings  were  determined  for  each  question.  Results  of  the 
mean  calculation  are  contained  in  Appendix  E.  The  PRR 
question  that  had  the  lowest  mean  value  was  identified  as 
the  question  that  should  receive  the  most  emphasis  when  per¬ 
forming  a  foreign  PRR;  the  question  receiving  the  highest 
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mean  value  is  the  question  that  should  receive  the  least 
emphasis  when  performing  a  foreign  PRR. 

Second  Primary  Finding 

The  F-16  Multinational  Coproduction  Program  tasked 
foreign  coproducers  with  a  "build  to  print"  operation  and 
no  design  or  development  responsibilities.  Therefore, 
emphasis  during  the  PRRs  on  these  firms  was  placed  on  readi¬ 
ness  for  a  production  operation  rather  than  design  and 
development  tasks.  Responsibility  for  design  and  develop¬ 
ment  rested  with  the  domestic  contractor  and  was  treated 
in  PRRs  of  the  domestic  contractor's  plants. 

This  indicates  a  significant  shift  in  emphasis  from 
PRRs  performed  on  domestic  contractors  but  not  an  unexpected 
shift.  "Coproduction  will  likely  be  a  frequent  method  of 
weapons  system  acquisition  in  the  future  [28:3]."  Arrange¬ 
ments  will  allow  foreign  contractors  the  same  "build  to 
print”  production  operations  as  is  presently  experienced  in 
the  F-16  progr£un.  Thus,  this  shift  in  emphasis  is  a  signifi¬ 
cant  difference  between  PRRs  performed  on  domestic  contrac¬ 
tors  and  those  performed  on  foreign  coproducers. 

Third  Primary  Finding 

Several  respondents  emphasized  that  the  list  of  25 
AFSCR  84-2  questions  was  not  inclusive  of  all  areas  that 
should  be  reviewed  in  a  foreign  PRR.  Most  comments  indicated 
special  attention  must  be  given  to  the  traditions,  values. 
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amd  cultures  of  the  foreign  country,  and  the  effect  of  these 
items  on  the  labor  force.  Another  area  that  needs  special 
attention  is  the  guidelines  affecting  transportation  of 
material  and  resources  between  foreign  countries.  A  full 
discussion  of  these  and  other  comments  is  provided  later  in 
this  chapter. 


Research  Hypothesis  II 

PRRs  performed  on  foreign  firms  have  different 
areas  of  emphasis  than  those  performed  on  domestic 
firms . 

Primary  Findings 

Brechtel  and  Lathrop's  1976  thesis,  "A  Comparative 
Analysis  of  the  Application  of  Production  Readiness  Reviews" 
(4),  studied  PRRs  that  had  been  performed  on  domestic  defense 
contractors.  Through  interviews  with  ten  AF  production  man¬ 
agement  experts,  Brechtel  and  Lathrop  determined  that  the 
experts  did  agree  on  the  relative  importance  of  all  25 
AFSCR  84-2  questions.  The  mean  rank  was  computed  for  each 
question  and  the  questions  then  assembled  in  order  of  their 
relative  mean  ranks  (4:144-147).  The  experts  generally 
agreed  that  the  questions  at  the  top  of  the  list  should 
receive  the  greatest  emphasis  during  a  domestic  PRR,  and 
those  at  the  bottom  of  the  list  should  receive  the  least 
emphasis.  The  rankings  found  by  Brechtel  and  Lathrop  are 
shown  in  Appendix  C. 
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In  this  thesis,  the  researchers  have  replicated  the 
procedures  used  by  Brechtel  and  Lathrop  and  have  assembled 


a  list  of  the  same  25  questions  as  ranked  by  ten  foreign 
PRR  experts.  This  list  is  shown  in  Appendix  C.  As  noted 
in  the  previous  chapter,  the  25  questions  were  drawn  from 
the  1971  AFSCR  84-2.  Use  of  the  questions  is  necessary  to 
ensure  validity  of  comparison  between  the  foreign  and  domes¬ 
tic  PRR  experts. 

To  test  the  hypothesis  that  PRRs  performed  on  foreign 
firms  have  different  areas  of  emphasis  than  PRRs  performed 
on  domestic  firms,  the  correlation  between  the  rankings  of 
each  of  the  above  lists  was  studied  using  the  Spearman  Rank 
Correlation  Coefficient  Test.  The  Kendall  Rank  Correlation 
Coefficient  Test  was  also  performed  for  verification.  High 
positive  correlation  would  indicate  there  is  little  differ¬ 
ence  in  the  areas  of  emphasis  between  the  two  lists.  Large 
negative  correlation  would  indicate  there  is  a  strong 
inverse  relationship  between  areas  of  emphasis.  If  this 
were  true,  those  areas  rated  least  important  in  domestic 
PRRs  would  be  most  in^ortant  in  foreign  PRRs. 

First  Primary  Finding 

The  null  hypothesis  (Hg)  for  the  correlation  test 
was  that  differences  do  exist  between  areas  of  emphasis; 
thus,  the  F-16  foreign  PRR  experts  ranked  the  questions  dif¬ 
ferently  than  did  the  domestic  experts  in  Brechtel  and 


Lathrop  study.  The  alternate  hypothesis  (H^^)  states  that 
there  exists  no  difference  between  areas  of  emphasis,  thus 
the  questions  were  assigned  the  same  rank  by  both  groups. 

As  discussed  in  the  following  sections,  the  data 
supported  differences  do  exist  between  areas  of  emphasis 
of  domestic  and  foreign  PRRs. 


Spearman-Tau  Reuik  Correlation  Coefficient  test. 

This  test  is  a  measure  of  the  degree  of  association  between 
the  sample  pairs  of  observations.  For  this  test,  the  data 
were  placed  in  X  sample  ranks  and  Y  Scunple  ranks.  The  X 
ranking  and  Y  ranking  represent  the  prioritized  rankings  of 
the  25  AFSCR  84-2  PRR  questions.  The  X  ranking  is  provided 
by  Brechtel  and  Lathrop  (4:144-147)  and  the  Y  ranking  is 
provided  by  the  foreign  PRR  experts  ranking  in  this  study.  A 
comparison  of  the  two  remkings  is  shown  in  Appendix  D. 

The  procedure  used  in  conducting  the  Spearman  Test 
was  covered  in  two  different  texts:  one  by  Siegel  (17:202), 
and  the  other  by  Gibbons  (9:275).  The  differences  of  each 
line  of  the  X  and  Y  rauikings  are  computed,  squared,  and 
summed.  If  the  relationship  between  the  two  sets  of  ranks 
were  perfect,  every  difference  would  be  zero.  The  larger 
the  difference,  the  less  perfect  must  be  the  association  of 
the  two  rankings.  A  restatement  of  the  hypotheses  is  in 
order  at  this  time.  The  null  hypothesis  (Hq)  for  the  corre¬ 
lation  test  is  that  differences  do  exist  between  areas  of 


emphasis;  thus,  the  F-16  foreign  PRR  experts  ranked  the 
questions  differently  than  the  domestic  experts  in  Brechtel 


and  Lathrop's  study.  The  alternate  hypothesis  (H^^)  states 
that  there  exists  no  difference  between  areas  of  emphasis; 
thus,  the  questions  were  ranked  the  same  by  both  groups. 

The  formula  used  to  compute  the  coefficient  was  as 

follows : 


T 


S 


N 

6  1  D.  ^ 
i=l  ^ 

-  N 


where 

=  The  difference  between  the  two  ranks 
N  *  Number  of  ranked  variables 
N 

I  Directs  one  to  sum  the  squares  of  differences 
i=l  between  the  two  ranks 

From  the  data. 


25 

1  D. 2  =  2630 
i=l  ^ 

T  =  1  -  6(2630) 

®  (25) ’  -  25 

=  -.011538 

The  value  of  the  coefficient  of  rank  correlation  defined  by 
Spearman's  Tau  is  always  equal  to  or  between  +1.00  and  -1.00 
with  the  interpretation  of  +1.00  as  being  the  maximum 
possible  agreement  or  perfect  agreement.  The  -1.00  value 


indicates  an  inverse  correlation  or  that  of  perfect  disa¬ 
greement.  Hence,  the  absolute  value  of  the  Spearman  coeffi¬ 
cient  of  1.00  reflects  perfect  correlation,  and  the  sign  of 
the  number  indicates  the  type  or  direction  of  correlation. 
When  the  coefficient  is  equal  to  zero,  there  is  no  correla¬ 
tion  and  hence  neither  agreement  nor  disagreement.  The 
results  of  this  test  indicate  that  there  is  very  little 
correlation,  if  any,  and  that  correlation  is  negative. 

A  more  precise  conclusion  can  be  reached  through  the 
use  of  the  table  of  critical  values  of  the  Spearman's  rank 
correlation  coefficient  (2:186).  If  the  observed  absolute 
value  for  Spearman's  Tau  equals  or  exceeds  the  value  in  the 
table,  then  the  observed  value  is  significant  at  the  speci¬ 
fied  level  indicated.  For  N=25  and  a  level  of  significance 
of  .05,  the  value  obtained  from  the  published  table  for  a 
two-tailed  test  is  0.400.  Since  the  observed  absolute  value 
for  Tau  was  .011538  and  this  value  does  not  exceed  or  equal 
the  tabled  value,  the  conclusion  is  to  accept  Hq.  The 
acceptance  of  the  null  hypothesis  concludes  that  the  rankings 
of  the  25  PRR  questions  by  the  two  groups  of  experts  are 
different. 

Since,  in  this  test  N  was  larger  than  ten,  the  sig¬ 
nificance  of  the  obtained  Spearman's  Tau  may  be  tested  using 
the  t-distribution.  The  Tau  was  transformed  into  a 
^“Statistic  using  the  following  formula: 


where 


Tg  =  -.011538 


=  -.55338 


The  t-distribution  table  (2:82)  shows  that  for  the  degrees 
of  freedom  equal  to  N-2=23  and  the  significance  level  of  .05 
(for  a  two-tailed  test)  the  critical  value  was  2.069;  Since 
the  computed  absolute  t  value  of  .055338  is  less  than  the 
critical  value,  the  conclusion  is  to  accept  the  null  hypoth¬ 
esis  (Hq)  that  the  rankings  were  unrelated.  This  test  gives 
credence  and  affirms  the  results  of  the  first  test  on  this 
data. 

For  another  verification  of  the  results  obtained 
from  the  nonparametric  Spearman's  Tau,  one  addition  para¬ 
metric  statistical  test  was  performed — the  Kendall  Rank 
Correlation  Coefficient. 

The  Kendall  Rank  Correlation  (Tau) .  The  Kendall  test 
(17:213;  9:284)  is  another  measure  of  correlation  between 
the  two  rankings  of  PRR  questions  by  the  experts.  It  is 
applicable  in  exactly  the  same  sampling  and  inference 
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situations  as  the  Spearman  test.  The  Kendall  Tau  statistic 
measures  association  in  a  slightly  different  way,  and  hence, 
in  most  cases  produces  a  numerical  value  different  from  the 
Spearman's  results.  Both  tests  utilize  the  Scune  £unount  of 
information  in  the  data,  and  thus,  both  have  the  same  power 
to  detect  the  existence  of  correlation  in  the  population. 

The  value  for  Tau  is  obtained  by  arranging  one  ranking  in 
numerical  order  and  then  examining  the  corresponding  rank 
values  for  the  second  ranking.  The  procedure  considers  all 
possible  pairs  of  rauik  values  in  the  second  ranking,  adding 
to  the  value  of  Tau  for  values  which  are  in  natural  order 
and  subtracting  for  pairs  out  of  order.  If  the  rankings  are 
in  perfect  agreement,  the  value  of  Tau  is  1.00,  indicating 
the  maximum  possible  agreement.  If  the  rankings  are  in 
reverse  numerical  order,  the  value  of  Tau  is  -1.00.  The 
Kendall  Tau  is  a  function  of  the  minimiun  number  of  inter¬ 
changes  of  remks  necessary  to  transform  the  second  ranking 
into  the  same  order  as  the  first  ranking  and  may  be  considered 
as  a  coefficient  of  disarray  between  the  two  rankings.  The 
procedure  and  calculations  for  this  statistical  test  are 
given  in  Appendix  G.  The  calculated  Tau  was  equal  to  -0.14. 

The  specific  test  of  the  null  hypothesis  is  performed 
by  con^aring  a  value  of  S  for  the  rank  order  comparison  to  a 
critical  value  of  fl  obtained  from  a  normal  distribution  teUsle. 
The  critical  S  value  identifies  the  allowable  probekbility  of 


42 


rejecting  the  null  hypothesis  when  the  null  hypothesis  is 
true. 

To  transform  the  Tau  value  to  a  0  value,  the  follow¬ 
ing  formula  was  used: 


Tau 


/2(2N  +  5) 
/  ^N(N-l) 


In  this  formula,  Tau  and  N  are  defined  as  they  have  been 
used  previously.  The  denominator  in  this  formula  is  the 
standard  deviation  of  Tau.  Therefore, 

S  = 

/Tuimrs 

/  9(i5) (24) 

=  -0.980909 


The  decision  rule  for  hypothesis  testing  is  based  on 
a  significance  level  of  .05.  An  inverse  relationship  or 
positive  relationship  is  considered  equally  conclusive 
for  rejecting  the  null  hypothesis  of  no  agreement.  The  test 
becomes  a  two-tailed  statistical  test  for  a  nonnal  distri¬ 
bution.  Any  computed  value  of  8  which  is  equal  to  or 
greater  than  1.96  or  equal  to  or  less  them  -1.96  will  cause 
the  null  hypothesis  to  be  rejected.  Since  the  calculated 
value  is  between  the  limits,  the  conclusion  is  to  accept 
which  is  the  same  conclusion  reached  in  the  Spearman's  Tau 
test.  It  can  be  concluded  with  considerable  assurance  that 
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the  ranking  of  the  25  AFSCR  84-2  PRR  questions  by  the 
foreign  PRR  experts  does  not  agree  with  the  ranking  of  the 
same  questions  by  domestic  PRR  experts. 

Second  Primary  Finding 

These  tests  show  that  neither  positive  or  negative 
correlation  exists  between  the  two  ranked  lists.  The  ranks 
assigned  by  the  foreign  PRR  experts  are  different  than 
those  assigned  by  the  domestic  PRR  experts.  The  researchers 
investigated  the  nature  of  the  differences  and  observed 
several  significance  findings. 

Change  in  rank.  All  25  PRR  questions  received  two 
ranks.  One  was  the  rank  assigned  by  domestic  PRR  experts, 
the  other  by  foreign  PRR  experts.  The  change  in  ranks 
between  the  two  populations  can  provide  interesting  insight. 
The  change  in  rank  (D)  ranged  from  1  to  20.  Eight  questions 
had  a  D  r  5,  i.e.,  they  changed  positions  by  less  than  five 
places.  At  the  other  end  of  the  spectrum,  four  questions 
had  a  D  2  15,  or  changed  position  by  at  least  15  places. 

The  questions  chamging  the  least  number  of  places 
may  be  typified  by  their  concern  for  production  capability, 
plannii^g  and  tooling.  Question  numbers  are  assigned  by  1971 
AFSCR  84-2.  Specifically,  the  following  four  questions 
changed  only  one  position:  Question  number  (3)  .  .  .  pro¬ 
duction  tooling  has  been  demonstrated  .  .  .,  (6)  .  .  .  ad¬ 
vanced  production  planning  has  been  accomplished  .  .  . , 
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(12)  .  .  .  impact  of  changes  on  production  have  been 
assessed  .  .  (19)  .  .  .  manufacituring  capabilities  have 

been  technically  evaluated. 

The  inference  here  is  that  the  importance  of  review¬ 
ing  these  areas  does  not  change  when  performing  a  PRR  on  a 
foreign  firm  rather  than  a  domestic. 

The  questions  changing  the  greatest  number  of  places 
may  be  typified  by  their  concern  for  the  very  basics  of  a 
producer's  organization;  is  there  adequate  skilled  labor  and 
is  the  organization  designed  in  a  way  to  allow  accomplishment 
of  production.  Specifically,  the  questions  changing  20  and 
18  positions,  respectively,  were  question  numbers  (23)  .  .  . 
availability  of  adequate  labor  skills  has  been  studied  .  .  . , 
and  (24)  .  .  .  contractor  is  adequately  organized  to  accom¬ 
plish  production.  Both  these  questions  shifted  from  very 
low  importance  in  the  domestic  ranking  to  very  great  impor¬ 
tance  in  the  foreign  ranking.  One  question  shifted  20 
places  in  the  opposite  direction.  Question  (2)  .  .  .  engi¬ 
neering  and  development  problems  have  been  resolved  .  .  . , 
was  found  to  be  very  important  domestically  but  of  little 
importance  in  a  foreign  PRR.  As  noted  earlier,  this  is  not 
suirprising  since  the  major  U.S.  contractors  held  all  respon¬ 
sibility  for  design  and  development. 

Top  five  questions.  The  foreign  PRR  experts  indicated 
the  five  PRR  questions  to  receive  the  most  en^hasis  were  (in 
order) :  (9)  .  .  .  assure  readiness  of  production  equipment. 


methods,  and  labor  .  .  (6)  .  .  .  advanced  production 

planning  accomplished  .  .  (18)  .  .  .  adequate  management 

information  systems  are  present  .  .  (23)  .  .  .  availa¬ 

bility  of  adequate  labor  skills  has  been  studied  .  . 

(24)  .  .  .  contractor  is  adequately  organized  to  accomplish 
production  ....  The  foreign  PRR  ranking  has  only  one 
question  in  common  with  the  top  five  of  the  domestic  PRR 
experts,  question  (6)  dealing  with  advanced  production 
planning. 

Conspicuously  absent  from  the  foreign  Top  5  are 
four  questions  dealing  with  engineering,  design,  and  develop¬ 
ment.  Specifically,  the  questions  removed  from  the  foreign 
Top  5  include  (13)  .  .  .  results  of  technical  reviews  and 
status  of  unresolved  problems  have  been  studied  .  .  . , 

(1)  .  .  .  engineering  design  is  practical  and  producible  .  .  ., 
(22)  .  .  .  engineering  problems  have  been  resolved  .  .  ., 

(5)  .  .  .  any  significant  design  changes  required  have  been 
implemented  .... 

It  is  evident  the  F-16  foreign  coproducers  were  not 
responsible  for  design  and  development  duties  in  the  program, 
and  the  PRRs  were  adjusted  to  accommodate  this.  There  are, 
however,  a  number  of  shifts  in  emphasis  which  do  not  deal 
with  engineering  and  development  duties.  Foreign  PRRs 
emphasized  basic  business  needs  such  as  labor  supply,  organi¬ 
zational  design,  and  production  technology.  These  areas 
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received  much  greater  emphasis  in  the  foreign  firm  than  they 
would  have  received  in  the  domestic  contractor's  plant. 


Summary  of  Primary  Findings 
The  findings  of  Research  Hypothesis  I  and  II  conclude 
that  Production  Readiness  Reviews  on  foreign  coproducers 
require  special  emphasis  on  certain  areas  and  that  further¬ 
more,  the  25  PRR  questions  in  AFSCR  84-2  may  be  ranked  to 
reflect  the  relative  in^ortance  of  each  area.  Secondly,  the 
areas  of  emphasis  for  a  foreign  PRR  are  different  than  the 
areas  of  emphasis  for  a  domestic  PRR.  There  are  areas  that 
must  be  carefully  studied  when  doing  a  foreign  PRR  that  do 
not  need  close  scrutiny  when  performing  a  domestic  PRR.  The 
areas  requiring  the  greatest  emphasis  during  a  foreign  PRR 
pertain  to  the  coproducer's  organization,  his  production 
planning,  his  management  information  system,  production 
tooling,  and  availability  of  skilled  labor.  A  significant 
difference  is  the  lack  of  attention  to  engineering,  design, 
and  development  responsibilities.  Also  of  interest  here  is 
the  fact  that  meuiy  foreign  PRR  experts  felt  the  list  of 
questions  in  the  1971  AFSCR  84-2  was  not  inclusive  of  all 
areas  to  be  reviewed  in  a  successful  foreign  PRR.  The  list 
should  be  augmented  by  additional  questions  to  suit  the 
specific  system  or  contractor.  These  points  and  additional 
data  and  observations  have  been  discussed  in  the  next  section. 

Corollary  Findings. 
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Corollary  Findings 


This  research  effort  was  directed  to  the  collection, 
analysis,  and  interpretation  of  data  to  determine  if  dif¬ 
ferences  exist  between  areas  of  emphasis  in  foreign  and 
domestic  PRRs,  auid  then  to  investigate  the  nature  of  those 
differences.  Based  on  this  investigation,  the  researchers 
sought  to  determine  if  the  PRR  process  was  indeed  appliccdsle 
to  the  foreign  coproducer  in  a  multinational  program.  Pre¬ 
ceding  sections  presented  primary  findings  in  support  of 
Hypothesis  I  and  II.  In  addition  to  primary  findings,  other 
interesting  findings  evolved  from  the  reviews  of  DOD  PRR 
publications,  collected  data,  and  interviews  with  PRR  partici¬ 
pants.  T!;ese  additional  findings  provided  added  support  for 
the  conclusions  reached  in  Hypothesis  I  and  II. 

Most  of  these  findings  were  spawned  by  the  responses 
to  the  questionnaire  (Appendix  A) .  In  addition  to  the 
ranking  of  the  25  PRR  questions,  the  interviewee  was  asked 
five  questions  pertaining  to  his  experiences  in  the  PRR 
process.  Each  of  the  multipart  questions  covered  one  area 
of  the  PRR.  As  a  result,  the  corollaxry  findings  have  been 
presented  in  the  seune  sequence  as  were  the  five  questions 
that  uncovered  them. 

First  Corollary  Finding  (Question  1) 

An  investigation  was  made  to  determine  the  proper 
timing  of  the  PRR.  The  regulation  (AFSCR  84-2)  calls  for 
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termination  of'  the  PRR  at  the  time  of  DSARC  III,  yet  there 
may  be  cases  where  it  is  sovind  management  to  continue  the 
process  beyond  this  point.  The  first  question  focused  on 
the  timing  of  the  PRR  on  the  foreign  coproducer. 

All  ten  respondents  said  that  the  PRRs  and  their 
follow-ups  extended  beyond  the  DSARC  III  milestone.  Further¬ 
more,  all  said  that  these  post-DSARC  III  reviews  generated 
facts  which  improved  the  production  process  most  of  the  time. 
Comments  by  the  respondents  indicated  that  follow-ups  beyond 
this  point  were  essential  to  good  management,  but  that  they 
should  be  limited  to  action  items  on  specific  areas  of 
concern . 

The  respondents  were  equally  divided  when  asked  if 
they  spent  the  greater  amount  of  their  time  and  effort  before 
or  after  DSARC  III.  This  could  be  a  result  of  when  they 
joined  the  program  or  an  indication  that  a  significant  amount 
of  time  is  spent  after  DSARC  III  Milestone.  All  felt  that 
the  time  spent  before  DSARC  III  was  always  productive  because 
the  decision  to  "go"  with  foreign  production  came  from  the 
briefing  prepared  with  these  PRR  facts.  The  PRRs  revealed 
serious  problems  that  had  to  be  eliminated  prior  to  the 
decision.  Had  these  problems  not  been  detected  and  corrected, 
a  negative  impact  on  production  would  have  resulted. 

The  respondents  unanimously  felt  that  foreign  PRRs 
should  extend  beyond  the  DSARC  III  milestone.  PRRs  are 
representative  of  sound  management  and  will  promote  better 


communication  between  the  foreign  firm  and  the  U.S.  govern¬ 
ment.  Most  felt  that  the  PRRs  should  extend  beyond  this 
point  in  the  form  of  follow-ups  on  areas  where  pre-DSARC  III 
problems  existed.  They  should  be  more  informal  than  the 
pre-DSARC  III  PRRs  and  should  eventually  give  way  to  normal 
surveillance.  Lastly,  PRRs  should  be  initiated  any  time  a 
major  engineering  change  or  schedule  slippage  occurs. 

In  short,  the  first  corollary  finding  was  that 
experts  agree  that  the  PRRs  should  not  be  limited  to  a  pre- 
DSARC  III  activity.  They  are  a  useful  management  tool  that 
provided  benefits  both  before  and  after  the  decision  mile¬ 
stone,  euid  must  be  continued  in  the  form  of  follow-ups  after 
the  production  readiness  decision  is  made. 

Second  Corollary  Finding 

All  foreign  firms  reviewed  during  the  PRR  process 
were  subcontractors  to  some  U.S.  contractor,  in  most  instan¬ 
ces  the  prime.  It  would  seem  that  the  prime  would  have 
enough  vested  interest  in  the  foreign  coproducer  to  perform 
the  same  type  of  inspection  as  a  PRR.  The  second  question 
focused  on  whether  the  U.S.  contractor  could  have  accom¬ 
plished  the  same  objectives  as  the  government  PRR  teeun.  If 
they  could,  it  would  be  logical  (and  cost  beneficial)  to 
make  the  prime  responsible  for  the  production  readiness  of 
the  subcontractor. 
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The  ten  experts  felt  that  the  prime  had  slightly 
less  interest  than  the  government  SPO  team  in  production 
areas  excunined  by  the  PRR.  Comments  indicated  the  prime  did 
not  rely  upon  deliveries  from  coproducers  to  meet  its  ovm 
schedule,  and  often  did  not  receive  any  of  the  foreign 
coproducer's  output.  Furthermore,  one  expert  felt  that  the 
prime  did  not  share  all  available  information  with  the  for¬ 
eign  coproducer  when  solving  troublesome  problems. 

Six  out  of  ten  experts  felt  the  prime  could  not  have 
performed  the  PRR  with  as  much  success  as  the  government 
directed  SPO  team.  Their  reasoning  was  the  government 
approaches  the  coproducer  in  the  role  of  cm  interested  cus¬ 
tomer,  while  the  prime  approaches  as  a  partner.  Because  of 
the  difference  in  perspective,  the  prime  will  not  put  as 
much  effort  into  the  PRR  or  cover  as  broad  a  spectrum  of 
production  areas.  One  expert  noted,  however,  there  is  a  very 
strong  marketing  incentive  for  the  prime  to  ensure  the 
coproducers  are  successful  and  stay  on  schedule. 

In  summary,  a  majority  of  experts  felt  the  primes 
should  not  be  charged  with  responsibility  for  the  PRR  process 
They  lack  the  interest  of  a  customer  and  are  inclined  to 
treat  the  foreign  coproducer  as  a  future  competitor  rather 
than  partner.  This  results  in  reluctance  to  share  technology 
and  know  how  during  the  review  process. 
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Third  Corollary  Finding 

The  official  objective  of  the  PRR  is  to  gather  facts 
pertaining  to  the  production  readiness  of  the  foreign  firm 
and  to  verify  that  the  production  phase  may  be  entered  with¬ 
out  incurring  unacceptcdsle  risks.  The  third  question  focused 
on  the  extent  of  the  expert's  involvement  with  the  foreign 
coproducer.  Did  they  only  collect  information  and  report 
back  to  the  DSARC  III,  or  did  they  in  fact  help  solve  trouble¬ 
some  production  problems? 

All  respondents  felt  they  both  gathered  information 
and  solved  problems  to  some  degree.  Three  went  further  and 
stated  the  two  tasks  were  inseparable.  Responses  were  evenly 
divided  among  those  spending  more  time  gathering  information, 
and  those  spending  more  time  solving  problems.  Sometimes 
the  problems  with  the  foreign  coproducers  were  apparent.  In 
these  cases,  the  coproducer  relied  upon  the  government  for 
technical  assistance  and  problem  solution.  The  most  commonly 
used  procedure  by  the  F-16  SPO  was  to  identify  actual  or 
potential  problems,  esteUslish  action  items  or  correct  the 
problems,  and  then  perform  follow-up  PRRs  to  ensure  the 
discrepancies  were  resolved.  To  siunmarize,  the  experts  felt 
that  identifying  the  problem  is  the  first  step  of  problem 
solving,  so  the  two  tasks  are  inseparable.  The  PRR  teams 
did,  however,  go  beyond  their  "fact  finding"  charter  and 
provide  valu^d^le  technical  assistance  and  problem  solving 
skills  to  the  foreign  contractor. 
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The  foreign  coproducer's  attitude  towards  the  PRR 
may  be  summed  up  with  this  response  to  one  of  the  questions. 
"The  Europeans  were  worried  about  us  until  they  found  out 
we  were  not  trying  to  run  their  business  and  that  we  were 
there  as  partners  to  make  the  program  work."  Another 
response  gives  us  a  similar  view.  "Initially  they  (foreign 
contractor)  felt  imposed  upon  (by  PRR) ,  but  as  we  worked 
with  them  most  of  them  became  more  receptive  when  they  found 
out  that  we  were  not  there  to  criticize  but  to  help  them. " 
Seven  out  of  ten  experts  felt  that  after  overcoming  initial 
resistance,  the  PRRs  were  welcomed  by  the  foreign  contractor. 

Oddly,  only  one  expert  felt  the  PRR  was  inadequately 
marketed  to  the  foreign  contractor.  All  noted  the  initial 
resistance  of  the  contractor  but  felt  that  no  additional 
preparation  was  needed  prior  to  the  initial  PRR.  This 
indicates  perhaps  that  the  advance  marketing  of  the  PRR 
reduced  resistance  to  a  degree.  Beyond  this  point  resistance 
could  only  be  reduced  by  working  side  by  side  with  the 
government  PRR  team.  The  experts  agreed  that  resistance  broke 
down  after  the  PRR  teeuns  got  into  the  contractor's  facilities 
and  began  work. 

The  third  corollary  finding  was  that  the  experts  feel 
they  did  more  than  gather  facts  during  the  PRR  process,  and 
that  the  advance  preparation  of  the  foreign  coproducer  for 
the  PRR  was  adequate.  The  government  teams  acted  as  technical 
advisors  and  problem  solvers  to  some  degree,  and  after 


overcoming  initial  resistance,  they  were  welcomed  by  the 
foreign  firm. 

Fourth  Corollary  Finding 

Question  four  offered  the  experts  an  opportunity  to 
identify  and  comment  on  the  areas  that  required  special 
emphasis  when  performing  a  PRR  on  a  foreign  contractor.  Each 
respondent  was  asked  to  identify  three  areas  that  they  felt 
were  significantly  more  important  in  the  foreign  plant  than 
in  the  domestic  contractor’s  plant.  Many  areas  were  singled 
out  as  requiring  special  attention.  Among  these  areas, 
several  were  repeatedly  mentioned  by  the  ten  foreign  PRR 
experts . 

Ladjor  environment.  The  area  most  often  mentioned 
concerned  the  labor  environment  in  the  foreign  country.  The 
PRR  teaun  must  be  thoroughly  fcuniliar  with  local  IcUaor  unions, 
labor  legislation,  and  the  work  ethic  of  the  particular 
country.  Problems  in  any  of  these  areas  can  have  an  acute 
impact  on  the  production  readiness  and  production  capability 
of  the  foreign  contractor. 

Many  of  the  respondents  cited  examples.  In  the  case 
of  one  foreign  firm,  stedsle  and  full  employment  was  more 
important  than  avoiding  schedule  slippages.  Some  firms 
operate  on  a  one  shift  basis  and  are  unwilling  to  increase 
working  hours  to  adjust  schedule  or  production  changes. 
Overtime  is  virtually  unheard  of.  Management  and  labor 
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unions  permit  surprise,  one  day  strikes  when  the  weather  is 
nice.  In  short,  the  schedules  and  goals  of  the  production 
line  are  not  sacred.  The  PRR  team  must  carefully  study  the 
impact  this  type  of  work  ethic  can  cause  on  the  production 
line. 

Less  middle  management.  Another  area  repeatedly 
mentioned  by  the  ten  experts  was  the  organizational  design 
of  the  foreign  contractor.  There  is  a  conspicuous  lack  of 
middle  management  in  the  foreign  firm,  and  the  general 
manager  absorbs  much  of  the  responsibility  for  handling  all 
phases  of  the  production  line.  Because  there  is  less  manage¬ 
ment,  there  is  less  specialized  reporting  to  the  general 
manager.  The  PRR  team  may  not  be  presented  facts  and  numbers 
in  the  same  detail  they  could  expect  in  a  domestic  firm.  The 
general  manager  may  be  the  single  point  of  contact  for  the 
PRR  team.  His  span  of  control  may  be  large,  and  he  relies 
upon  foreman  and  floor  supervisors  to  keep  production  moving. 
In  this  case,  the  PRR  team  member  must  possess  great  technical 
skill  as  he  must  often  detect,  compare  and  analyze  his  own 
data  directly  from  the  floor  of  the  production  line.  The 
lack  of  middle  managers  means  that  many  of  the  problems  in 
the  production  line  may  go  unnoticed  until  the  PRR  team 
arrives.  The  PRR  team  members  need  high  technical  skill  to 
pinpoint  trouble  areas  in  the  foreign  coproducer's  production 
line. 
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Longer  lead  times.  The  ten  foreign  PRR  experts  felt 


that  foreign  coproducers  require  longer  lead  times  than 
domestic  contractors.  Several  factors,  they  felt,  contrib¬ 
ute  to  this  situation.  In  many  cases,  the  contractor  is 
unfamiliar  with  the  new  technology  required  for  production 
and  there  is  a  very  slow  learning  curve.  More  importantly, 
and  harder  to  overcome,  was  the  absence  of  a  sense  of  urgency 
in  some  contractors.  This  again  reflects  the  impact  of  the 
labor  environment  and  work  ethic  encountered  by  the  PRR 
experts.  The  lack  of  urgency  coupled  with  the  delays  of 
handling  new  technology  mean  one  thing  to  the  PRR  team;  the 
time  required  to  complete  a  task  is  greater  for  a  foreign 
coproducer  than  for  a  domestic  contractor.  The  PRR  team  must 
be  well  aware  of  the  longer  lead  times  required  in  the  for¬ 
eign  sector  and  recognize  the  slow  reaction  of  the  foreign 
coproducer  to  schedule  chcmges  and  surges. 

Customs  of  the  foreign  country.  The  ten  foreign 
PRR  experts  felt  that  dealing  with  the  foreign  contractor 
requires  extreme  sensitivity  to  the  customs  and  culture  of 
the  country  itself.  Language  differences  present  a  frus¬ 
trating  hurdle,  but  a  society's  customs  can  present  even  a 
more  tenacious  obstacle.  The  experts  found  they  could  not 
charge  into  a  plant,  perform  the  PRR,  then  leave.  Most 
foreign  PRRs  required  a  great  deal  more  diplomacy  than 
domestic  contractors.  Several  days  may  be  devoted  to  intro¬ 
ductions  and  making  acquaintances.  The  foreigners  practice 
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much  of  the  old  world  protocol  with  which  contemporary 
domestic  contractors  long  ago  dispensed.  The  PRR  teams  may 
have  to  survive  several  days  of  decoriam  and  protocol  before 
they  can  get  down  to  business.  In  addition,  to  language  and 
protocol,  other  facets  of  the  foreign  country's  culture  may 
affect  the  contractor's  ability  to  do  business  within  the 
coproduction  framework.  National  holidays  may  affect  the 
coproducer's  adherence  to  schedule.  The  foreign  country's 
relations  with  a  neighboring  country  may  inhibit  the  trans¬ 
port  of  crucial  materials  and  resources  across  borders.  A 
difference  in  standards  includes  much  more  than  a  comparison 
of  metric  units  versus  the  fauniliar  mile,  foot,  and  inch  of 
U.S.  measure.  Is  their  standard  workday  as  long  as  a 
domestic  workday?  Is  the  standard  technical  expertise  of  a 
worker  the  same  as  that  of  a  domestic  worker? 

The  PRR  team  members  must  do  a  careful  study  of  the 
customs  and  cultures  of  the  foreign  country  they  are  to  visit. 
They  should  be  familiar  with  any  aspect  of  customs  that  may 
impact  the  production  readiness  of  the  coproducer.  Further, 
they  must  be  prepared  to  observe  the  customs  themselves 
including  an  abundance  of  protocol  at  each  meeting  with  the 
foreign  contractor. 

Quality  control.  Responses  of  the  ten  PRR  experts 
indicated  that  the  foreign  contractor  required  particular 
scrutiny  in  the  area  of  quality  control.  In  many  cases,  the 
contractor  was  overwhelmed  by  new,  high  technology  production 


57 


processes  and  had  yet  to  develop  a  sound  quality  control 
program  for  the  production  line.  Foreign  quality  control 
programs  were  not  as  well  developed  as  domestic  quality 
control  programs.  In  this  area,  the  PRR  team  is  able  to 
provide  technical  advice  and  help  the  contractor  understand 
the  importance  2uid  necessity  of  a  solid,  comprehensive  QC 
progreun.  The  foreign  PRR  team  should  give  special  attention 
to  quality  control  systems,  especially  in  the  case  of  copro¬ 
duction  where  foreign  produced  parts  may  be  installed  on 
U.S.  aircraft. 

Other  areas.  Several  other  areas  were  identified 
by  the  experts  as  requiring  special  enphasis  and  considera¬ 
tion.  Most  of  these  additional  areas  were  also  brought  to 
the  surface  by  the  ranking  of  the  25  PRR  questions.  Receiving 
the  greatest  emphasis  was  production  planning  and  organization 
design.  Typical  comments  point  out  that  planning  is  more 
critical  for  the  foreign  contractor  because  often  the  output 
from  his  production  line  becomes  the  input  to  a  different 
line,  often  in  a  different  country.  The  sequencing  and 
scheduling  of  events  is  also  critical  to  prevent  backlogs  or 
overloads  in  scheduled  deliveries.  The  emphasis  on  these 
areas  has  been  pointed  out  in  primary  findings  stemming  from 
Hypothesis  II.  The  PRR  team  should  look  at  these  areas  more 
carefully  during  a  foreign  PRR. 
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Fifth  Corollary  Finding 

The  Department  of  Defense  has  recently  become 

involved  in  the  PRR  arena.  DOD  Product  Engineering  Services 

Office  (DOD/PESO)  completed  a  study  of  PRRs  which  included 

a  note  about  PRRs  performed  on  foreign  contractors. 

The  PRRs  at  foreign  svibcontractor  plants  benefitted 
the  PMO  (Project  Management  Office)  and  contractors  in 
other  ways.  The  PRR  teams  provided  consultant  services 
and  aided  technology  transfer  by  identifying  deficiencies 
in  the  subcontractor  production  plan.  The  PRR  also 
highlighted  differences  in  business  practices  between 
domestic  and  foreign  defense  contractors  .... 

Those  experienced  with  PRRs  at  foreign  contractor's 
facilities,  found  differences  in  culture  or  language 
which  compounded  the  problems  usually  encountered  on 
domestic  PRRs.  The  form  of  government,  Ministry  of 
Defense  Structure,  industry/government  relationship,  and 
Icdsor/management  relationship  are  all  differences  that 
must  be  addressed  during  planning  and  execution  of  the 
PRR  [27:16]. 


Summary  of  Corollary  Findings 

Responses  from  the  questionnaire  and  investigation 
of  related  data  have  led  to  the  following  corollary  findings: 

1.  The  ten  experts  all  feel  future  foreign  PRRs 
should  extend  beyond  the  DSARC  III  milestone  in  the  form  of 
follow-up  reviews. 

2.  A  majority  of  the  ten  experts  feel  that  the 
prime  should  not  be  charged  with  the  responsibility  for  per¬ 
forming  PRRs  on  foreign  coproducers. 

3.  Nine  out  of  ten  experts  felt  the  PRR  was  ade¬ 
quately  marketed  to  the  foreign  contractor.  They  felt  the 
SPO  PRR  te2un8  acted  as  technical  advisors  and  problem 
solvers,  and  after  overcoming  initial  resistimce,  they  were 
welcomed  by  the  foreign  firm. 

4.  The  experts  felt  certain  areas  not  covered  by 
the  AFSCR  84-2  questions  deserve  special  attention  during 
a  foreign  PRR.  These  additional  areas  included: 


a.  The  labor  environment  of  the  country 

b.  The  typical  lack  of  middle  management 

c.  The  longer  lead  times  required  by  foreign 

contractor 

d.  Customs  of  the  foreign  country 

e.  The  lack  of  well  developed  quality  control 

progreuns 

5.  The  DOO  has  reviewed  PRRs  and  made  some  observa¬ 
tions  on  foreign  PRRs.  They  note  that  teeun  members  should 
be  well  versed  in  tradition  and  culture  of  the  foreign 
participants. 


Summary 

The  primary  findings  of  this  chapter  have  emerged 
from  the  emalysis  of  Research  Hypotheses  I  and  II.  The 
corollary  findings  resulted  from  analysis  of  questionnaires, 
interviews,  and  study  of  PRR  related  material.  The  findings 
indicate  that  PRRs  are  imperative  when  dealing  with  a  foreign 
contractor.  In  the  next  chapter,  the  conclusions  reached 
from  these  findings  are  discussed. 
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Chapter  5 


CONCLUSIONS  AND  RECOMMENDATIONS 
Overview 

This  chapter  discusses  conclusions  associated  with 
the  findings  of  the  previous  chapter.  Primary  findings  are 
related  to  the  two  major  objectives  of  this  study;  corollary 
findings  were  discovered  from  additional  data  collected  from 
research,  interviews,  and  questionnaires.  The  researchers' 
major  objectives  were  to  determine  if  foreign  PRRs  require 
special  attention  to  certain  areas  and  to  determine  if  these 
areas  of  emphasis  were  different  for  foreign  and  domestic 
PRRs. 

The  primary  conclusions  evolved  from  statistical 
analysis  of  the  ten  experts'  ranking  of  25  standard  PRR 
questions.  The  corollary  conclusions  evolved  as  more  data 
was  assembled  ^uld  reviewed.  The  corollary  findings  are  pre¬ 
sented  to  provide  a  more  thorough  picture  of  the  foreign 
PRR  and  the  special  challenges  it  presents. 

Research  Hypothesis  I 

Different  PRR  teams  concentrated  on  similar  areas 
of  emphasis  when  performing  PRRs  on  foreign  firms  in  the 
F-16  program. 
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Hypothesis  Support 

Research  Hypothesis  l  was  supported.  The  ten  F-16 
foreign  PRR  experts  agreed  on  the  relative  rankings  of  the 
25  questions.  The  Friedman  Two-Way  Analysis-of-Variance-by- 
Ranks  test  (17:166)  showed  that  differences  do  exist  between 
the  mean  ranks  for  the  25  questions.  Thus,  the  ten  experts 
felt  that  certain  questions  were  more  important  than  others 
when  performing  a  foreign  PRR. 

General  Conclusions 

The  primary  conclusion  resulting  from  the  test  of 
Research  Hypothesis  I  was  that  certain  areas  are  more  impor¬ 
tant  than  others  when  performing  a  foreign  PRR.  Thus,  the 
first  research  objective  achieved  by  the  discovery  that  dif¬ 
ferent  PRR  teams  concentrated  on  similar  areas  of  emphasis 
when  performing  PRRs  on  foreign  firms  in  the  F-16  progreun. 
Comments  from  the  respondents  indicated  these  areas  required 
more  attention  due  to  the  unique  challenges  presented  by  the 
multinational  coproduction  program  and  by  European  industry. 
All  experts  felt  that  their  experiences  were  not  characteris¬ 
tic  of  the  F-16  program  alone  but  that  similar  challenges 
would  be  encountered  any  time  the  United  States  deals  with  a 
European  contractor.  The  conclusion  was  that  in  future 
coproduction  programs,  foreign  PRR  teams  should  concentrate 
on  certain  areas  of  emphasis.  To  determine  if  these  areas 
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of  emphasis  are  the  same  as  those  of  a  domestic  PRR, 

Research  Hypothesis  II  was  developed. 

Research  Hypothesis  II 

PRRs  performed  on  foreign  firms  have  different  areas 
of  emphasis  than  PRRs  performed  on  domestic  firms. 

Hypothesis  Support 

Research  Hypothesis  II  was  supported.  The  mean  ranks 
for  the  25  questions  of  the  foreign  PRR  experts  do  not  agree 
with  the  mean  ranks  of  Brechtel  and  Lathrop's  domestic 
experts.  The  Spearman-Tau  Coefficient  of  Correlation  Test 
(17:279)  shows  there  was  no  agreement  between  the  two  rank¬ 
ings  of  the  25  questions.  Thus,  the  questions  that  are  most 
inportant  during  a  foreign  PRR  are  not  the  same  as  the  ques¬ 
tions  most  important  during  a  domestic  PRR. 

General  Conclusions 

The  primary  conclusion  resulting  from  the  test  of 
Research  Hypothesis  II  was  that  the  areas  of  emphasis  for 
a  foreign  PRR  are  different  than  the  areas  of  emphasis  for  a 
domestic  PRR.  Thus,  the  second  research  objective  was 
achieved  by  the  discovery  that  foreign  PRR  teeuns  should 
concentrate  on  areas  different  than  areas  of  concentration 
for  domestic  PRR  teams.  Comments  from  the  ten  experts 
revealed  that  European  industry  presents  unique  problems  and 
challenges  not  present  when  dealing  with  U.S.  contractors. 
These  unique  problems  are  present  throughout  European  industry 
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and  are  not  confined  to  the  F-16  coproducers.  Further, 
the  multinational  coproduction  program  placed  additional 
requirements  upon  the  foreign  firms ,  compounding  the  PRR 
challenge.  The  conclusion  was  that  in  the  future,  certain 
areas  will  require  special  emphasis  when  performing  a  PRR  on 
a  foreign  firm;  areas  that  would  not  require  as  much  emphasis 
if  a  domestic  firm  was  involved.  Additionally,  when  a  for¬ 
eign  contractor  is  also  a  coproducer,  other  areas  need 
emphasis  due  to  the  demanding  requirements  of  a  multinational 
coproduction  progreun.  Areas  requiring  the  greatest  attention 
during  a  foreign  PRR  are  discussed  in  the  previous  chapter, 
and  are  briefly  recounted  here.  The  top  five  questions  were 
the  five  questions  with  the  highest  mean  ranks  (ranks  1-5) . 

Top  five  questions.  The  top  five  questions  clearly 
indicated  that  attention  must  be  focused  on  the  foreign 
contractor's  ability  to  plan  the  production.  Adequate 
planning  and  preparation  is  crucial,  especially  in  a  copro¬ 
duction  effort.  A  delay  in  the  production  of  a  single  com¬ 
ponent  by  a  single  contractor  could  affect  the  delivery 
schedule  of  the  overall  system.  The  foreigners  lack  exper¬ 
ience  in  advance  production  planning  of  high  technology 
systems  cuid  conponents.  Often,  they  are  unfeuniliar  with  the 
technology  emd  with  the  rigidity  of  the  production  schedule. 
Thus,  a  thorough  review  of  the  following  questions  is  para¬ 
mount  to  successful  foreign  PRRs.  (Question  numbers  are 
those  assigned  by  1971  AFSCR  84-2). 
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Question  9.  "Assurance  of  readiness  of  manufacturing 
and  production  equipment,  methods,  facilities  ...  is 
available. " 

Question  6.  "Adequate  advanced  production  planning 
has  been  accomplished  .  .  .  . " 

Question  18.  "...  systems  for  providing  timely 

status  information  are  effective." 

Question  23.  "Availability  of  production  labor  skill 
requirements  has  been  assessed  and  their  acquisition  ade¬ 
quately  planned." 

Question  24.  "Contractor  is  adequately  organized 
to  accomplish  production  requirements." 

It  is  noteworthy  that  only  one  of  these  questions 
(number  6)  was  in  the  top  five  questions  identified  by 
Brechtel  and  Lathrop  as  being  most  impbrtant  in  a  domestic 
PRR.  There  was  a  fundcunental  shift  in  emphasis  towards 
basic  production  planning,  la±)or  availability,  and  manage¬ 
ment  feedback  systems.  Questions  in  the  domestic  top  five 
that  dropped  out  of  the  foreign  top  five  concerned  engineer¬ 
ing  design  and  development.  Further,  the  domestic  PRK 
experts  tended  to  de-emphasize  the  questions  in  the  foreign 
top  five,  ranking  three  of  those  at  the  low  ranks  of  18,  23, 
and  24. 

These  observations  led  to  the  conclusion  that  foreign 
PRR  teams  must  not  presume  the  foreign  contractors  are 
familiar  with  state-of-the-art  technology  and  production 
processes.  In  many  cases,  the  foreign  firm  is  not  exposed 
to  the  rapidly  changing  environment  that  characterizes  U.S. 
industry.  For  this  reason,  the  foreign  PRR  team  must  "get 
back  to  basics"  that  can  be  assumed  in  good  order  in  a 
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domestic  contractor.  The  PRR  team  member  may  quickly  find 
himself  in  the  role  of  a  management  consultant  providing 
technical  advice  and  solving  troublesome  problems  for  the 
foreign  contractor.  The  knowledge  and  expertise  of  the  PRR 
tecim  member  is  of  critical  importance.  Often,  he  must  not 
only  recognize  the  problem,  but  he  must  have  the  answer  as 
well. 

The  1971  AFSCR  84-2  list  of  25  questions  was  not 
comprehensive  for  a  foreign  PRR,  and  as  listed,  the  order  of 
the  questions  does  not  reflect  the  relative  importance  of 
each  question  when  performing  a  foreign  PRR.  The  1971 
format  tends  to  imply  a  ranking  of  relative  importance  that 
does  not  agree  with  the  ten  foreign  PRR  experts '  opinions 
of  the  relative  importance  of  each  question. 

Summary  of  Primary  Conclusions 

The  following  conclusions  were  reached  from  the 
primary  findings  of  Research  Hypothesis  I  and  ZI: 

1.  Certain  areas  are  more  important  than  others 
when  performing  foreign  PRRs. 

2.  These  areas  of  emphasis  are  different  for  PRRs 
performed  on  foreign  contractors  than  for  PRRs  performed  on 
domestic  contractors. 

3.  As  listed  in  the  1971  AFSCR  84-2,  the  questions 
do  not  reflect  relative  importance  for  foreign  PRRs. 

4.  Foreign  PRRs  require  special  attention  to  the 
basics  of  production  planning  and  call  for  technical  exper¬ 
tise  on  the  part  of  the  PRR  team  member. 
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Corollary  Conclusions 

This  section  of  the  research  addresses  the  conclu¬ 
sions  drawn  from  the  five  corollary  findings  discussed  in 
the  previous  chapter.  As  mentioned  before,  most  of  the 
corollary  findings  and  hence,  these  conclusions,  were 
spawned  by  the  responses  to  the  written  part  of  the  ques¬ 
tionnaire  (Appendix  H) .  In  addition  to  the  questionnaire, 
interesting  findings  evolved  from  the  review  of  DOD  PRR 
publications,  collected  data,  and  interviews  with  PRR  partic¬ 
ipants  . 

First  Corollary  Conclusion 

While  all  regulations  call  for  the  PRR  to  terminate 
at  DSARC  III,  all  ten  foreign  PRR  experts  indicated  that  the 
foreign  PRRs  and  their  follow-ups  extended  beyond  this  mile¬ 
stone.  They  generally  felt  that  follow-ups  beyond  this 
point  were  essential  to  good  management  but  should  be  limited 
to  previously  identified  action  items  or  specific  areas  of 
concern.  The  researchers  concluded  that  foreign  PRRs  should 
not  be  limited  to  a  pre-DSARC  III  activity.  They  should 
continue  beyond  DSARC  III  because  they  are  a  useful  manage¬ 
ment  tool  that  will  yield  benefits  both  before  and  after  the 
decision  point. 

Second  Corollary  Conclusion 


Although  all  foreign  coproducers  are  subcontractors 
to  some  U.S.  contractor,  the  U.S.  firm  does  not  necessarily 


show  the  same  interest  in  the  broad  spectrum  of  areas  covered 
by  the  PRR.  The  conclusion  reached  by  the  researchers  was 
that  the  primes  should  not  be  charged  with  responsibility  for 
performing  the  PRRs.  This  responsibility  should  remain  with 
the  government  who  assumes  the  role  of  a  very  interested  and 
concerned  customer. 

Third  Corollary  Conclusion 

The  foreign  PRRs  did  much  more  than  generate  facts 
for  the  production  decision  of  DSARC  III.  All  ten  respon¬ 
dents  felt  they  both  generated  information  and  solved  problems 
to  some  degree.  In  most  cases,  the  F-16  PRR  teams  went 
beyond  their  "fact  finding"  charter  and  provided  valuable 
technical  assistance  and  problem  solving  skills  to  the  for¬ 
eign  contractor.  The  researchers'  conclusion  was  that 
foreign  PRRs  require  more  than  mere  fact  finding.  The  PRR 
teams  must  be  prepared  to  get  deeply  involved  in  the  trouble¬ 
shooting  and  problem  solving  areas  during  the  PRR. 

Fourth  Corollary  Conclusion 

The  PRR  teams  encountered  initial  resistance  from 
the  foreign  coproducers,  but  resistance  gave  way  to  coopera¬ 
tion  when  coproducers  realized  the  teams  were  not  trying  to 
run  their  business  and  were  partners  in  the  coproduction 
program.  When  the  foreigners  realized  the  SPO  teams  were  not 
'here  to  criticize  but  to  help,  they  beceune  receptive.  A 
■M-ority  of  ten  experts  felt  the  initial  resistance 
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encountered  was  not  serious.  Nine  out  of  ten  experts  felt 
the  PRR  was  adequately  marketed  to  the  foreign  coproducer. 

The  researchers'  conclusion  was  that  the  PRRs  were  adequately 
marketed  to  the  foreign  contractors.  The  working  relation¬ 
ship  between  teams  and  coproducers  improves  after  the 
intangible  bond  of  trust  is  formed,  but  no  amount  of  addi¬ 
tional  marketing  could  eliminate  this  need  to  "check  each 
other  out." 

Fifth  Corollary  Conclusion 

Corollary  conclusion  five  through  nine  pertain  to 
written  comments  made  by  the  ten  experts  in  response  to 
question  four  in  the  questionnaire.  Their  written  comments 
and  the  associated  findings  provide  the  basis  for  these  con¬ 
clusions. 

The  foreign  labor  environment  was  repeatedly  noted 
by  the  experts  as  an  area  requiring  special  attention. 

Labor  unions,  labor  legislation,  skill  levels,  and  the  work 
ethic  of  the  populace  all  make  the  foreign  labor  environment 
different  than  that  of  the  United  States.  The  PRR  team  must 
carefully  study  and  paint  an  accurate  portrait  of  the  labor 
environment.  An  improper  estimate  of  labor  can  lead  to 
acute  problems  in  the  production  process.  The  researchers' 
conclusion  was  that  the  foreign  labor  environment  is  dis¬ 
tinctly  different  than  the  domestic  labor  environment  and  the 
PRR  teeun  must  carefully  study  the  impact  of  leUaor  on  a  firm's 
production  readiness. 
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sixth  Corollary  Conclusion 


The  ten  experts  witnessed  a  lack  of  middle  manage¬ 
ment  in  the  foreign  contractor's  organizations.  A  general 
manager  would  typically  absorb  the  responsibility  for  all 
phases  of  production.  Because  there  is  less  management, 
there  is  less  reporting  of  information  to  the  general  man¬ 
ager.  The  researchers'  conclusion  was  that  due  to  less 
middle  management  in  the  foreign  firm,  the  PRR  team  must 
possess  a  sharp  eye  for  detail  and  detect  production  problems 
not  apparent  to  the  coproducer.  The  PRR  team  members  will 
find  this  to  a  much  greater  extent  with  a  foreign  contractor 
than  with  a  domestic  contractor. 

Seventh  Corollary  Conclusion 

Several  factors  necessitate  longer  lead  times  for 
foreign  contractors  than  for  domestic  contractors.  New 
technology,  retraining  requirements,  the  local  work  ethic, 
the  lack  of  any  sense  of  urgency,  and  the  detailed  coordina¬ 
tion  required  for  a  coproduction  progreun  are  all  factors 
that  make  the  time  required  to  complete  the  task  longer  for 
the  foreign  contractor  than  for  the  domestic.  The  researchers 
conclusion  was  that  foreign  coproducers  react  more  slowly  to 
changes  than  domestic  coproducers,  and  that  longer  lead  times 
are  required  when  dealing  with  the  foreign  firm. 


Eighth  Corollary  Conclusion 

While  U.S.  firms  are  accustomed  to  dealing  with  the 
government  cuid  PRR  tecuns  can  get  to  work  quickly,  the  foreign 
coproducer  requires  more  attention  to  protocol  and  decorum. 
Dealing  with  the  foreign  contractor  requires  extreme  sensi¬ 
tivity  to  the  cultures  and  tradition  of  the  country  itself. 
Language  is  only  the  most  obvious  of  many  differences  between 
countries  and  cultures.  PRR  teams,  as  visitors  in  a  foreign 
country,  must  be  aware  of  and  practice  all  local  protocol 
habits.  The  researchers  concluded  that  most  foreign  PRRs 
will  require  more  diplomacy  amd  protocol  than  domestic  PRRs. 
Domestic  contractors  are  more  eager  to  "get  to  it,"  and  have 
dispensed  with  lengthy  introduction  and  familiarization 
routines  that  can  cause  delays. 

Ninth  Corollary  Conclusion 

Important  in  any  production  process,  quality  control 
and  standardization  are  critically  important  in  a  multina¬ 
tional  coproduction  program.  Parts  and  assemblies  from  20 
different  contractors  in  four  different  countries  must  meet 
together  on  the  assembly  line  and  fit  together  perfectly. 
Responses  of  the  ten  PRR  experts  indicated  quality  control 
was  an  area  requiring  special  emphasis  during  a  foreign  PRR. 
The  underdevelopment  of  the  foreign  quality  control  system 
may  be  contributed  to  by  the  lack  of  middle  management  and 
thus,  the  lack  of  any  specialized  reporting.  The  researchers' 
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conclusion  was  that  most  foreign  quality  control  progrcims 
are  not  as  well  developed  as  domestic  quality  control  pro¬ 
grams  and  the  foreign  PRR  team  must  give  special  attention 
to  this  area. 

Tenth  Corollary  Conclusion 

The  Air  Force  has  been  conducting  PRRs  for  over  ten 
years  as  compared  with  other  services  which  have  only 
recently  begun  the  practice  of  PRRs.  AFSCR  84-2,  issued  in 
November  1971,  has  been  the  forerunner  of  most  regulations 
concerning  Production  Readiness  Reviews.  The  Department  of 
Defense  has  recently  published  DOD  Directive  5000.38, 
Production  Readiness  Reviews,  and  the  other  services  have 
published  their  own  PRR  regulations.  There  is  no  doubt  that 
each  service  will  continue  performing  PRRs  in  the  future, 
and  that  inevitably,  PRRs  will  be  performed  on  foreign  con¬ 
tractors.  There  will  be  new  experiences  with  each  new  for¬ 
eign  PRR,  and  the  lessons  learned  will  be  of  value  to  all 
services.  The  researchers'  conclusion  was  that  each  service 
will  perform  foreign  PRRs  in  the  future  and  that  valuable 
lessons  may  be  learned  from  each  experience. 

Summary  of  Conclusions 

The  researchers  have  found  that  Production  Readiness 
Reviews  are  applicable  to  foreign  coproduction  programs,  and 
they  should  continue  in  the  future.  The  following  primary 
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and  corollary  conclusions  were  drawn  from  the  collection 
and  analysis  of  data. 


Primary  Conclusions 

1.  In  future  multinational  coproduction  programs, 
certain  production  areas  should  receive  more  emphasis  than 
others  during  a  foreign  PRR. 

2.  These  areas  requiring  special  emphasis  are  dif¬ 
ferent  for  PRRs  performed  on  a  foreign  firm  than  for  PRRs 
performed  on  a  domestic  firm. 

3.  As  listed  in  1971  AFSCR  84-2,  the  PRR  questions 
do  not  reflect  the  relative  emphasis  each  should  receive 
during  a  foreign  PRR. 


Corollary  Conclusions 

1.  Foreign  PRRs  should  not  terminate  at  the  DSARC 
III  milestone  and  should  continue  in  the  form  of  follow-ups 
on  specific  areas  of  concern. 

2.  The  prime  contractor  should  not  be  charged  with 
the  responsibility  of  performing  PRRs  on  foreign  subcon¬ 
tractors. 

3.  In  addition  to  gathering  facts  for  the  DSARC  III 
briefing,  foreign  PRR  teeuns  must  also  get  involved  in 
troubleshooting  and  problem-solving  with  the  foreign 
coproducer . 

4.  Foreign  PRRs  are  adequately  marketed  to  the  for¬ 
eign  coproducers. 

5.  The  labor  environment  in  the  foreign  country  is 
different  than  that  in  the  U.S.  and  must  be  carefully 
studied  by  the  foreign  PRR  team. 

6.  The  foreign  coproducer  typically  has  less  middle 
management  than  his  domestic  counterpart.  This  presents 
unique  challenges  for  the  foreign  PRR  team. 

7.  Foreign  coproducers  react  more  slowly  to  changes 
than  domestic  coproducers,  and  longer  lead  times  are  required 
when  dealing  with  the  foreign  firm. 
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8.  Most  foreign  PRRs  require  more  diplomacy  and 
protocol  than  domestic  PRRs. 

9.  Most  foreign  quality  control  programs  are  not 
as  well  developed  as  domestic  quality  control  programs. 

10.  Each  component  of  the  DOD  will  perform  foreign 
PRRs  in  the  future  and  valuable  lessons  may  be  learned  from 
each  experience. 

Recommendations  for  Future  Foreign  PRRs 

As  a  result  of  the  primary  findings  and  conclusions 
reached  in  testing  Research  Hypotheses  I  and  II,  and  the 
corollary  findings  and  conclusions,  the  following  recommenda¬ 
tions  are  made  for  future  foreign  PRRs. 

The  list  of  25  standard  PRR  questions  in  the  1971 
AFSCR  84-2  (20:3-4)  provided  a  tool  for  valid  comparison 
between  foreign  and  domestic  PRR  experts.  Brechtel  and 
Lathrop  used  an  identical  list  in  their  1976  thesis.  Using 
Brechtel  emd  Lathrop 's  methodology,  the  same  list  was  pro¬ 
vided  to  the  foreign  PRR  experts  in  this  study.  Although 
the  1971  AFSCR  84-2  has  been  superseded  and  the  25  questions 
are  now  presented  in  different  format,  the  use  of  the  earlier 
list  of  questions  is  valid  for  comparison  purposes.  In  fact, 
the  1981  AFSCR  84-2  (21)  has  addressed  one  primary  conclusion 
reached  in  this  study.  The  third  primary  conclusion  notes 
that  as  listed  in  the  1971  regulation,  the  PRR  question  did 
not  reflect  the  relative  emphasis  each  should  receive  during 
a  foreign  PRR.  The  1981  regulation  subjectively  groups  the 
questions  into  eight  areas.  Presentation  of  the  PRR  criteria 
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in  this  fashion  is  more  useful  to  the  PRR  planner.  Still, 
the  foreign  PRR  planner  has  no  indication  of  which  of  these 
subject  areas  need  more  emphasis  during  a  foreign  PRR. 

First  recommendation.  Primary  conclusions  reached 
during  the  testing  of  Hypotheses  I  and  II  indicate  that  areas 
requiring  special  emphasis  are  different  for  PRRs  performed 
on  foreign  firms  than  for  PRRs  performed  on  domestic  firms. 
The  particular  areas  requiring  special  emphasis  during  a 
foreign  PRR  are  discussed  in  the  Findings  chapter  and  in 
Appendix  H.  The  researchers  recommend  that  future  foreign 
PRR  programs  be  developed  around  the  prioritized  areas  of 
emphasis  furnished  in  Appendix  C.  The  questions  at  the  top 
of  the  list  indicate  areas  requiring  the  most  emphasis  during 
a  foreign  PRR.  Lower  ranking  questions  indicate  areas 
requiring  less  emphasis. 

Second  recommendation.  Another  specific  recommenda¬ 
tion  for  future  foreign  PRR  programs  is  that  foreign  PRRs 
should  extend  beyond  the  DSARC  III  milestone  in  the  form  of 
follow-ups  on  specific  areas  of  concern.  Deficiencies  in 
production  readiness  must  be  identified  and  tracked  until 
corrected.  This  may  often  require  follow-up  visits  to  the 
foreign  contractor  after  the  production  decision  is  made  at 
the  DSARC  III  milestone.  Further,  any  change  in  schedule  or 
design  may  warrant  a  visit  by  the  PRR  teeun  to  the  foreign 
contractor. 
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Third  recommendation.  The  researchers '  third  recom¬ 
mendation  is  that  U.S.  contractors  should  not  perform  PRRs 
on  their  foreign  subcontractors  in  a  coproduction  progreun. 

The  responsibility  for  the  PRR  should  remain  with  the  govern¬ 
ment  program  office.  The  U.S.  contractor  may  participate 
on  the  PRR  team  but  should  not  be  responsible  for  providing 
the  government  with  all  production  readiness  data  on  the 
foreign  contractor.  Only  the  government  itself  can  truly 
gather  objective  information  for  the  OSARC  III  decision  and 
fill  the  role  of  the  wary  buyer. 

Fourth  recommendation.  The  fourth  recommendation 
concerns  the  extent  of  the  PRR  team's  involvement  with  the 
foreign  contractor.  Teams  should  be  prepared  to  go  beyond 
the  fact-gathering  phase  of  involvement  and  must  often 
troubleshoot  and  help  solve  production  problems.  The  PRR 
personnel  and  foreign  contractors  form  a  team;  a  teeun  com¬ 
mitted  to  the  success  and  smooth  operation  of  the  production 
line.  This  often  requires  the  PRR  teeun  members  to  act  as 
technical  advisors  to  relieve  some  of  the  pressure  presented 
by  new  technology  and  production  processes. 

Fifth  recommendation.  The  fifth  recommendation  is 
that  foreign  PRR  teeuns  should  be  prepared  to  deal  with  cer¬ 
tain  problems  peculiar  to  foreign  industry.  These  areas  are 
not  cleeurly  addressed  in  the  1971  or  1981  AFSCR  84-2,  but 
their  consideration  can  enhance  a  foreign  PRR  team's  ability 
to  plan  and  execute  the  foreign  PRR.  The  following  six  areas 
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are  discussed  in  the  Conclusions  chapter  and  should  be  con¬ 
sidered  prior  to  conducting  a  foreign  PRR. 

1.  The  foreign  PRR  team  is  likely  to  encounter 
initial  resistance  from  the  foreign  contractor,  but  this 
resistance  will  give  way  to  receptiveness  when  the  foreign 
contractor  realizes  the  SPO  teams  are  there  to  help. 

2.  The  foreign  PRR  team  must  be  sensitive  to  the 
labor  environment  in  a  foreign  country.  Many  factors  affect 
the  skills,  availaibility ,  and  attitudes  of  the  labor  force. 

3.  The  foreign  PRR  team  must  be  prepared  for  less 
middle  management  in  the  foreign  firm.  They  must  expect 
less  specialized  reporting  and  less  management  information 
to  be  present  for  their  use. 

4.  The  foreign  PRR  teeun  must  recognize  that  longer 
lead  times  are  typically  required  for  any  adjustments  to  the 
production  process.  Foreign  firms  adapt  more  slowly  to 
change  than  domestic  firms. 

5.  The  foreign  PRR  teams  must  be  prepared  to  respect 
the  customs  and  traditions  of  the  host  country.  Most  foreign 
PRRs  will  require  more  diplomacy  and  protocol  than  domestic 
PRRs. 


6.  Most  foreign  quality  control  systems  are  not  as 
well  developed  as  domestic  programs  and  thus  deserve  the 
special  attention  of  the  PRR  tecun. 

Sixth  recommendation.  The  sixth  recommendation  is 
that  a  data  base  be  formed  to  store  information  gained 
during  foreign  PRRs.  Each  coR^onent  of  the  DOD  will  perform 
PRRs  in  the  future.  Valuedsle  lessons  will  be  learned  from 
each  experience,  as  well  as  detailed  information  cd>out  indi¬ 
vidual  contractors.  Each  service  component  has  an  organiza¬ 
tion  charged  with  responsibility  for  coordinating  PRR  efforts 
and  ensuring  they  are  accomplished.  In  the  case  of  the  Army, 
a  Production  Engineering  Services  Organization  (ARPESO)  was 
recently  esteUslished  within  the  Material  Development  and 
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Readiness  Conunand  (DARCOM) .  The  Navy  has  established 
NAVPESO  within  its  own  Material  Conunand.  Air  Force  Systems 
Command  manages  PRRs  for  the  Air  Force  through  the  Directorate 
of  Manufacturing  in  each  of  five  buying  divisions.  Each  of 
these  offices  assist  the  program  manager  in  the  planning 
and  execution  of  PRRs.  Additionally,  they  must  coordinate 
with  DPESO  on  all  PRR  matters.  DPESO  acts  as  the  coordi¬ 
nating  body  for  each  of  the  PRR  organizations  in  the  service 
components.  At  the  November  1980  DOD  PRR  conference,  it  was 
noted  that  the  defense  production  community  needs  a  PRR 
data  bank  to  include  lessons  learned  auid  contractor  assess¬ 
ments.  ARPESO,  with  its  associated  DARCOM  facilities,  was 
proposed  by  DPESO  to  study  the  possibility  of  establishing 
such  a  data  bank  (27:294).  DPESO  is  the  agency  most 
eligible  to  be  responsible  for  such  a  data  bank.  Executive 
mcuiagement  of  the  data  bank  can  be  delegated  to  one  of  the 
services.  It  is  ^recommended  that  this  data  bank  include  PRR 
data  on  foreign  contractors  as  well  as  domestic.  This 
information  would  be  invalu^J^le  to  the  PRR  teams  during  the 
planning  phase  for  a  foreign  PRR.  All  available  data  sources 
should  be  presented  to  the  SPO  teeun  for  their  utilization. 

Summary  of  Recommendations 
for  Future  Foreign  PRRs 

As  stated  earlier  in  this  study,  the  findings,  con¬ 
clusions,  and  recommendations  may  be  generalized  to  foreign 
firms  in  the  NATO  sphere  of  influence.  The  F-16  SPO  personnel 
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participated  in  PRRs  on  numerous  European  contractors,  but 
there  is  a  lack  of  experience  in  dealing  with  contractors 
in  other  geographic  sectors  of  the  world.  The  researchers 
feel  consideration  of  these  six  recommendations  during  future 
PRRs  on  European  contractors  is  necessary. 

Summary 

These  recommendations  for  future  foreign  PRRs,  along 


with  recommendations  for  future  research,  are  assembled  in 
Chapter  6. 


Chapter  6 


PINAL  RECOMMENDATIONS 

Overview 

As  discussed  in  the  previous  chapters,  multinational 
coproduction  and  U-S.  government  involvement  with  foreign 
contractors  will  be  increasingly  common  in  the  future  (28:3). 
The  risk  of  the  marketplace  increases  when  the  U.S.  looks 
overseas  for  procurement  of  system  and  subsystems.  The  PRR 
provides  a  means  for  the  U.S.  to  reduce  risk  when  dealing 
with  a  foreign  contractor.  The  F-16  multinational  coproduc¬ 
tion  program  has  illustrated  the  success  of  a  solid  PRR  pro¬ 
gram  (13:p. 3-ii-lC) .  Based  on  conclusions  reached  in  the 
preceding  chapter,  the  researchers  made  specific  recommenda¬ 
tions  for  consideration  when  planning  future  foreign  PRRs. 

Two  types  of  recommendations  have  surfaced:  first  are 
recommendations  for  management  action  during  future  foreign 
PRRs;  and  second  are  recommendations  for  future  research. 
Preceding  these  recommendations  is  a  discussion  of  the 
appliceUaility  of  the  recommendations  to  Air  Force  Systems 
Command  Regulation  84-2  (AFSCR  84-2)  and  a  recommendation 
concerning  a  supplement  to  the  regulation  addressing  foreign 
PRRs. 
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Air  Force  Systems  Command  Regulation  84-2 

The  Air  Force  has  been  conducting  PRRs  for  over  ten 
years.  AFSCR  84-2,  issued  in  November  1971,  has  been  the 
forerunner  of  most  regulations  concerning  Production  Readi¬ 
ness  Reviews.  The  Department  of  Defense  recently  published 
DODI  5000.38,  Production  Readiness  Reviews,  as  a  guide  to  all 
services  for  conducting  PRRs.  Since  its  issuance  in  January 
1979,  much  experience  has  been  gained  in  planning,  conducting, 
and  evaluating  PRRs.  The  Defense  Production  Engineering 
Services  Office  (DPESO)  is  the  issuing  agency  for  the  DOD 
regulation  and  the  focal  point  for  collection  of  PRR  experi¬ 
ences  and  information. 

In  January  1981,  Air  Force  Systems  Command  published 
a  rewrite  of  AFSCR  84-2.  The  new  regulation  supersedes  the 
November  1971  regulation  and  is  in  close  harmony  with  DODI 
5000.38.  In  fact,  the  1981  AFSCR  84-2  uses  the  Scune  criteria 
for  evaluating  production  readiness  as  does  the  TOD  regula¬ 
tion.  The  similarities  between  the  two  regulations  are  so 
apparent  that  the  1981  AFSCR  84-2  appears  to  have  been  a 
rewrite  of  DODI  5000.38,  not  of  the  1971  AFSCR  84-2. 

As  was  true  in  the  1971  AFSCR  84-2,  the  rewrite  makes 
no  mention  of  PRRs  performed  on  foreign  contractors  nor  any 
mention  of  PRRs  performed  on  firms  involved  in  a  multina¬ 
tional  coproduction  program.  Conclusions  reached  in  this 
research  effort  indicate  the  foreign  contractor  poses  unique 
challenges  to  the  PRR  team;  challenges  which  may  be  identified 
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and  which  will  aid  the  program  manager  when  planning  future 
foreign  PRRs.  Neither  the  1971  or  1981  AFSCR  84-2  addresses 
these  unique  challenges.  Thus,  the  conclusions  and  recom¬ 
mendations  in  this  research  effort  are  as  applicable  after 
the  rewritten  AFSCR  84-2  was  published  as  they  were  before. 

Supplement  to  AFSCR  84-2 

The  1981  AFSCR  84-2  does  not  list  25  standard  PRR 
questions  as  does  the  1971  regulation.  Instead,  the  ques¬ 
tions  are  grouped  together  in  eight  subject  areas  derived 
from  OODI  5000.38.  The  subject  areas  and  individual  criteria 
are  for  use  by  PRR  teams  in  determining  production  readiness. 
Program  managers  are  encouraged  to  develop  specific  criteria 
to  meet  situational  needs.  The  objective  is  to  adequately 
evaluate  production  readiness.  The  researchers  feel  the 
AFSCR  84-2  presentation  of  criteria  is  far  more  useful  than 
the  random  list  of  25  questions.  However,  the  new  regulation 
still  gives  no  attention  to  the  case  of  a  PRR  performed  on  a 
foreign  firm.  The  experiences  and  lessons  learned  from  pre¬ 
vious  PRRs  are  not  reflected  in  the  1981  regulation.  There 
is  no  doubt  the  nvunber  of  foreign  PRRs  will  increase  in  the 
future.  The  omission  of  instructions  concerning  foreign  PRRs 
is  significant.  The  researchers  feel  AFSCR  84-2  should 
include  discussion  of  foreign  PRRs,  thus  it  is  their  recom¬ 
mendation  that  AFSC  publish  a  supplement  to  the  1981  regula¬ 
tion.  The  supplement  should  specifically  address  the  case  of 
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the  foreign  PRR  and  identify  areas  requiring  special  emphasis 
and  management  attention. 


Recommendations  for  Management  Action 
As  a  result  of  the  primary  findings  and  conclusions 
reached  in  testing  Research  Hypotheses  I  and  II,  and  the 
corollary  findings  eind  conclusions,  the  following  recom¬ 
mendations  are  made  for  management  action  by  different  Air 
Force  and  DOD  agencies.  Once  again,  two  areas  of  recommenda¬ 
tions  have  been  made:  first  are  six  specific  recommendations 
for  management  action  during  future  foreign  PRRs;  and  second, 
recommendations  for  future  research  concerning  foreign  PRRs. 
The  six  specific  recommendations  were  presented  in  the 
previous  chapter.  The  action  office  for  implementing  the 
first  five  recommendations  will  be  HQ  AFSC's  coordinating 
body  for  the  Directorates  of  Manufacturing  in  each  of  AFSC's 
five  buying  divisions.  The  manufacturing  directorates  are 
the  agencies  responsible  for  ensuring  the  accomplishment  of 
PRRs  by  program  managers.  OPESO  will  be  the  action  office 
for  the  sixth  recommendation  as  it  is  most  eligible  to 
coordinate  the  PRR  data  from  each  service  con^nent. 

Recommendations  for  Future  Research 
The  use  of  PRRs  is  expanding  throughout  the  Depart¬ 
ment  of  Defense.  It  will  soon  be  present  during  weapons 
system  acquisitions  for  all  services  as  it  now  is  for  the 
Air  Force.  The  requirement  to  conduct  PRRs  as  set  forth  in 
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OODI  5000.38  will  prevail.  Moreover,  weapons  systems  will 
increasingly  be  the  product  of  a  cooperative  effort  among 
several  countries.  Multinational  coproduction  will  be  the 
rule  rather  than  the  exception  in  the  future.  The  require¬ 
ments  for  foreign  PRRs  will  increase  dramatically.  With  the 
increase  in  the  number  of  foreign  PRR  prograuns  availedsle  for 
study,  the  opportunities  for  meaningful  research  on  foreign 
PRRs  will  increase.  This  section  discusses  four  areas  for 
future  research  on  foreign  PRRs. 


Replication  of  This  Study 

As  noted  before,  the  conclusions  and  recommendations 
of  this  study  are  generalized  for  foreign  contractors  in  the 
NATO  sphere  of  influence.  These  contractors  and  their 
respective  countries  share  certain  characteristics  and 
commonalities.  The  seunple  reviewed  by  the  F-16  SPO  personnel 
was  a  cross  section  of  European  industry.  Based  on  this 
sample,  the  conclusions  indicate  expected  conditions  of  the 
European  industrial  environment. 

Each  area  of  the  world  displays  different  industrial 
characteristics.  As  esqierience  is  gained  in  other  sectors 
of  the  world,  those  areas  may  be  studied  through  the  per¬ 
sonnel  performing  foreign  PRRs  in  these  areas.  In  an  effort 
to  gather  information  about  foreign  contractors  in  other 
areas  of  the  world,  replication  of  this  study  is  recommended 
with  enphasis  on  other  non-European  areas  of  the  world. 


Foreign  Contractor  Viewpoint 

This  study  has  addressed  the  foreign  PRR  from  the 
government  point  of  view  only.  The  foreign  contractor  has 
many  perceptions  of  the  PRR  process  as  well.  It  has  been 
noted  that  foreign  contractors  are  initially  resistant  to 
the  foreign  PRR  te6uns.  This  resistance  later  yields  to 
cooperation  when  the  foreign  contractor  feels  more  certain 
of  the  U.S.  government  teeun’s  good  intentions.  A  question 
that  arises  here  is  whether  the  initial  resistance  received 
during  foreign  PRRs  is  as  great  as  resistance  received 
during  domestic  PRRs.  To  better  understand  the  contractor's 
point  of  view  and  perceptions  of  the  PRR  process,  future 
study  of  the  contractor's  viewpoint  is  recommended. 

Data  Base  for  PRR  Information 

As  the  number  of  foreign  PRRs  increase  and  as  all 
service  components  get  involved  in  the  review  process,  the 
information  and  facts  generated  by  PRRs  will  reach  enormous 
proportions.  Some  means  of  documenting  and  storing  "lessons 
learned"  and  contractor  information  must  be  created  on  a 
DOD-wide  basis.  The  data  bank  could  disseminate  information 
to  individual  agencies  as  they  prepared  for  foreign  PRRs. 

An  observation  made  at  the  DOD  PRR  Conference  in  November 
1980,  stated  "Experiences  gained  in  conducting  PRRs  to  date 
would  be  beneficial  to  others  holding  PRRs  in  the  future 
[27:293]."  Future  research  to  establish  some  sort  of  data 
bank  for  PRR  information  is  recommended. 
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Continuity  of  PRR  Personnel 

Comments  made  by  personnel  involved  and  related  to 
PRRs  indicated  the  PRR  process  is  hampered  by  the  continual 
reassignment  of  personnel.  A  major  acquisition  may  last 
several  years;  yet  PRR  personnel  may  be  reassigned  after  only 
one  or  two  years.  Continuity  of  personnel  is  important  due 
to  the  corporate  memory  the  individual  possesses,  and  the 
rapport  he  has  est^d^lished  with  the  contractor.  Additionally, 
continuity  of  personnel  would  allow  a  uniform  approach  to 
be  used  with  each  successive  PRR  on  a  particular  contractor. 
Research  on  the  feasibility  of  longer  duration  assignment 
for  PRR  personnel  is  recommended. 

Concluding  Observations 

Production  Readiness  Reviews  are  indeed  applicable 
to  multinational  coproduction  programs.  The  review  of  for¬ 
eign  contractors  involved  in  a  coproduction  process  should 
continue  in  the  future  for  a  number  of  reasons.  The 
researchers  feel  the  U.S.  government  is  destined  to  develop 
weapons  systems  with  other  friendly  covintries  in  a  coopera¬ 
tive  manner.  If  such  is  the  case,  good  management  dictates 
that  the  fitness  of  the  business  partner  be  checked  out. 

The  PRR  provides  a  "one-on-one"  consultation  between  the 
military  department  and  the  foreign  contractor.  This  con¬ 
sultation  cem  eliminate  numy  potential  problems  early  in  the 
prograuB  that  could  later  haunper  production  schedules  for  all 
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involved  parties.  The  PRR  also  reduces  the  uncertainty 
present  when  making  the  production  decision  at  DSARC  Mile¬ 
stone  III.  The  facts  generated  during  foreign  PRRs  were 
always  useful  in  making  this  production  decision.  The  Air 
Force  can  feel  confident  the  foreign  contractor  can  meet 
the  production  schedule.  PRRs  also  foster  good  strong  rela¬ 
tionships  between  the  government  and  the  foreign  contractor. 
When  this  rapport  exists,  the  foreign  contractor  is  more 
likely  to  easily  adapt  to  schedule  and  design  changes. 

It  is  hoped  that  this  research  study  will  aid  in  the 
planning,  execution  ,  and  results  of  future  foreign  PRRs  and 
that  it  may  provide  a  stepping  stone  for  future  research  in 
the  application  and  usefulness  of  the  PRR  process. 
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APPENDIX  A 

SAMPLE  QUESTIONNAIRE  WITH  CUMULATIVE 
RESPONSES 


89 


PRODUCTION  READINESS  REVIEW  (PRR) 
QUESTIONNAIRE 


This  questionnaire  is  in  two  parts.  The  first  part 
is  focused  on  the  relative  importance  of  the  twenty- five 
standard  PRR  questions  posed  in  Air  Force  Systems  Command 
Regulation  84-2  (AFSCR  84-2) .  The  second  part  focuses  on 
your  opinions  and  experiences  concerning  your  participation 
in  PRRs  on  foreign  firms. 

Please  indicate  the  number  of  PRRs  in  which  you  have 
participated  throughout  your  career.  Indicate  with  the 
niunerical  number  in  which  you  have  participated  by  category. 


Foreign  Domestic 


AIRFRAME  _  _ 

ENGINE  _  _ 

AVIONICS  _  _ 

MECHANICAL  SUBSYSTEMS  _  _ 

Totals  260  229 


PART  I 


This  part  of  the  questionnaire  aims  at  determining 
whether  a  standard  PRR  approach  can  be  developed  to  practi¬ 
cally  be  applied  to  various  foreign  contractors  in  a  major 
system  acquisition.  Additionally/  the  questionnaire  will 
determine  if  differences  exist  in  the  areas  of  emphasis  of 
PRRs  conducted  on  domestic  firms  and  foreign  firms.  Part  I 
consists  of  two  tasks. 

la:  Consider  a  priority  system  with  5  priority 
assignments  A  to  E.  An  assignment  of  A  indicates  highest 
priority  or  in^rtance,  while  an  assignment  of  E  indicates 
the  lowest  priority  or  least  importance.  Please  review  the 
following  twenty-five  standard  PRR  questions  (AFSCR  84-2) 
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and  assign  a  priority  to  each  question.  Please  assign  the 
priority  according  to  importance  when  performing  a  PRR  on 
a  foreign  contractor.^ 


STANDARD  AFSCR  84-2  PRR  QUESTIONS 


1.  Milestones  which  demonstrate  the  achievement  _ C 

of  a  practical  and  producible  engineering 

design  have  been  met. 

2.  Engineering  problems  encountered  during  devel-  _ C 

opment  have  been  resolved  with  appropriate 
tradeoffs  against  stated  operating  require¬ 
ments  so  that  production  costs/schedules  are 
optimized. 

3.  Critical  production  and  engineering  and  pro-  _ B 

duction  tooling  have  been  demonstrated  to 

prove  that  engineering  has  been  satisfac¬ 
torily  acconplished. 

4.  Acquisition  will  smoothly  transition  from  full-  _ C 

scale  development  to  production. 

5.  System  configuration  has  been  reviewed  to  _ C 

determine. if  any  significant  design  changes 

will  be  required  for  manufacturing. 

6.  Adequate  advance  production  planning  has  been  _ B 

accomplished  and  required  production  controls 
established  to  ensure  timely  production. 


^The  alpha  priority  assignments  were  converted  to 
numerical  assignments  for  computation  as  follows: 

A  *  1 
B  «  2 
C  »  3 
D  *  4 
E  ■  5 

To  decode  the  averages  back  to  alpha  symbols,  the  following 
was  used: 


A  -  1.01  -  1.75 
B  »  1.76  -  2.50 
C  «  2.51  -  3.25 
D  «  3.26  -  4.00 
E  -  4.01  -  4.75 
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7.  A  systematic  approach  to  standardization  has  _ E 

been  accomplished  in  the  design  process  and 

parts  selection  to  maximize  the  use  of  mili¬ 
tary  standard  components,  parts,  and  processes 
consistent  with  the  system  requirements. 

8.  Product  assurance  controls  and  tests  to  pre-  _ C 

vent  manufacturing  degradation  of  performance 
parameters  have  been  established. 

9.  Assurance  of  the  readiness  of  the  manufactur-  _ A 

ing  and  production  equipment,  methods,  facili¬ 
ties,  test  and  training  equipment,  and  status 

of  accessory  and  ancillary  items. 

10.  Plcuined  production  schedules  reflect  economy  _ C 

of  operations  and  minimize  financial  commit¬ 
ments  until  all  major  problems  have  been 

resolved. 

11.  A  thorough  assessment  of  the  make-or-buy  _ B 

structure  has  been  accomplished  and  proce¬ 
dures  exist  so  control  and  visibility  of  the 
vendors  and  subcontractors  can  be  effectively 
managed. 


12.  Change  activity  during  development  has  been  _ C 

evaluated  and  the  impact  of  outstanding  changes 

on  production  has  been  assessed. 

13.  Results  of  technical  reviews  and  the  produc-  C 

tion  impact  of  unresolved  problems  and  risks 

have  been  assessed. 

14.  Test  program  results  and  the  status  of  quali-  _ B 

fication  to  determine  production  risks  and 

risk  have  been  evaluated. 

15.  Specifications  and  drawings  have  been  reviewed  _ C 

to  assure  their  adequacy  for  the  planned  pro¬ 
duction  phase. 

16.  Application  of  production  tooling  and  test  _ E 

equipment  to  manufacturing  during  develop¬ 
ment  has  been  assessed  and  the  application 

of  same  to  the  production  phase  has  been 
defined. 
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17.  Material  management  system  for  determination  _ A 

of  requirements,  procurement,  receiving, 
inspection,  materials  handling  and  storage, 
inventory  control,  control  of  finished  goods, 
and  shipment  is  adequate. 


18.  Production  management  systems  used  for  pro-  _ A 

viding  management  with  timely  production 

status  information  are  effective. 

19.  Production  or  manufacturing  capabilities  of  _ A 

major  subcontractors  and  vendors  have  been 
technically  evaluated  and  found  adequate. 

20 .  Constraints  of  laboratory  or  model  shop  capa-  _ C 

bill ties  versus  quantity  production  require¬ 
ments  have  been  fully  considered. 

21.  Quality  controls  and  inspection  procedures  _ B 

have  been  established  for  materials  treatment 

or  processes  to  be  used  in  production. 

22.  Assessment  of  the  GFP  or  services  requirements,  _ C 

controls,  management,  and  availability  of 
suppliers  has  been  accomplished. 

23.  Availability  of  production  l^d>or  skill  require-  _ B 

ments  has  been  assessed  and  their  acquisition 
adequately  planned. 

24.  The  contractor  is  adequately  organized  to  _ C 

accomplish  the  production  requirements. 

25.  Planning  has  been  made  to  assure  timely  _ E 

release  of  manufacturing  instructions. 


lb:  Please  rank  the  twenty-five  PPvR  questions  in 
AFSCR  84-2  in  order  of  importance  to  a  foreign  PRR.  A  "one" 
should  be  assigned  to  the  PRR  question  that  should  have  the 
greatest  cunount  of  time,  effort,  and  resources  expended  for 
analysis  and  reporting,  and  a  "twenty-five"  should  be  assigned 
to  the  least  important  question.  Please  rank  the  questions 
according  to  how  you  feel  about  performing  a  PRR  on  a  foreign 
contractor . 
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STANDARD  AFSCR  84-2  QUESTIONS 


1.  Milestones  which  demonstrate  the  achievement  13 

of  a  practical  and  producible  engineering 

design  have  been  met. 

2.  Engineering  problems  encountered  during  devel-  22 

opment  have  been  resolved  with  appropriate 
tradeoffs  against  stated  operating  require¬ 
ments  so  that  production  costs/schedules  are 
optimized. 

3.  Critical  production  and  engineering  and  pro-  _ 8 

duction  tooling  have  been  demonstrated  to 

prove  that  engineering  has  been  satisfac¬ 
torily  accomplished. 


4.  Acquisition  will  smoothly  transition  from  full-  21 
scale  development  to  production. 

5.  System  configuration  has  been  reviewed  to  16 

determine  if  any  significant  design  changes 

will  be  required  for  manufacturing. 

6.  Adequate  advance  production  planning  has  been  _ 2 

accomplished  and  required  production  controls 
established  to  ensure  timely  production. 

7.  A  systematic  approach  to  standardization  has  25 


been  accomplished  in  the  design  process  and 
parts  selection  to  maximize  the  use  of  mili¬ 
tary  standard  components,  parts,  and  processes 
consistent  with  the  system  requirements. 

8.  Product  assurance  controls  and  tests  to  pre-  18 

vent  manufacturing  degradation  of  performance 
parameters  have  been  established. 

9.  Assurance  of  the  readiness  of  the  manufactur-  _ 1 

ing  and  production  equipment,  methods,  facili¬ 
ties,  test  and  training  equipment,  and  status 

of  accessory  and  ancillary  items. 

10.  Planned  production  schedules  reflect  economy  23 

of  operations  and  minimize  financial  commit¬ 
ments  until  all  major  problems  have  been 
resolved. 
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11.  A  thorough  assessment  of  the  make-or-buy 

structure  has  been  accomplished  and  proce¬ 
dures  exist  so  control  and  visibility  of  the 
vendors  and  subcontractors  can  be  effectively 
managed . 


12.  Change  activity  during  development  has  been  11 

evaluated  and  the  impact  of  outstanding  changes 

on  production  has  been  assessed. 

13.  Results  of  technical  reviews  and  the  produc-  15 

tion  impact  of  unresolved  problems  and  risks 

have  been  assessed. 

14.  Test  progr«un  results  and  the  status  of  quali-  19 

fication  to  determine  production  risks  and 

risk  have  been  evaluated. 

15.  Specifications  and  drawings  have  been  reviewed  17 

to  assure  their  adequacy  for  the  planned  pro¬ 
duction  phase. 

16.  Application  of  production  tooling  and  test  _ 9 


equipment  to  manufacturing  during  develop¬ 
ment  has  been  assessed  and  the  application 
of  same  to  the  production  phase  has  been 
defined. 

17.  Material  management  system  for  determination  _ 7 

of  requirements,  procurement,  receiving, 
inspection,  materials  handling  and  storage, 
inventory  control,  control  of  finished  goods, 

and  shipment  is  adequate. 

18.  Production  management  systems  used  for  pro-  _ 3 

viding  management  with  timely  production 

status  information  are  effective. 

19.  Production  or  manufactvuring  capabilities  of  _ 6 

major  subcontractors  and  vendors  have  been 
technically  evaluated  and  found  adequate. 

20.  Constraints  of  laboratory  or  model  shop  capa-  24 

bilities  versus  quantity  production  require¬ 
ments  have  been  fully  considered. 

21.  Quality  controls  and  inspection  procedures  12 

have  been  est^d}llshed  for  materials  treatment 

or  processes  to  be  used  in  production. 
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22.  Assessment  of  the  GFP  or  services  requirements, 
controls,  management,  and  availability  of 
suppliers  has  been  accomplished. 

23.  Availability  of  production  labor  skill  require-  _ 4 

ments  has  been  assessed  and  their  acquisition 
adequately  planned. 

24.  The  contractor  is  adequately  organized  to  _ 5 

accomplish  the  production  requirements. 

25.  Planning  has  been  made  to  assure  timely  10 

release  of  manufacturing  instructions. 


PART  II 


This  part  of  the  questionnaire  is  designed  to  collect 
information  about  your  experiences  as  a  participant  in  an 
F-16  PRR  on  a  foreign  contractor.  By  collecting  and  analyzing 
the  responses  of  many  participants  in  F-16  foreign  PRRs,  it 
is  hoped  that  information-  will  be  gathered  to  be  used  in 
future  PRR  planning.  The  questions  are  aimed  at  your  opinion 
of  the  PRR  process  and  how  well  it  works.  Do  not  feel  con¬ 
fined  by  the  responses  provided;  please  feel  free  to  add  any 
additional  comments  you  like  after  each  question.  It  is 
expected  your  thoughtful  responses  will  aid  in  the  timing  and 
execution  of  future  foreign  PRRs. 


1.  By  definition  in  AFSCR  84-2,  the  PRR  program  begins  when 
the  SPO  starts  planning  the  PRR,  and  terminates  at  the 
time  of  the  DSARC  III  review.  Although  there  is  no 
requirement  to  extend  the  PRR  process  beyond  the  DSARC  III 
milestone,  there  may  be  good  reason  to  continue  PRRs  and 
their  follow-ups  beyond  this  point. 

a.  Do  you  feel  the  foreign  PRRs  and  their  follow-ups  in 
which  you  participatea  terminated  at  the  DSARC  III 
milestone? 

Yes  0  No  10 


Comments : 
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b.  If  not,  was  the  greater  part  of  your  time  and  energy 
expended  before  the  DSARC  III  milestone,  or  in  the 
follow-ups  after  the  decision  point. 

Before  5  After  5 


Comments : 

c.  Did  the  PRR  in  which  you  participated  generate  facts 
that  were  useful  during  the  production  readiness 
decision? 


Please  circle  your 

response . 

None  of 

Some  of 

Half  of 

Most  of 

All  of 

the  Time 

the  Time 

the  Time 

the  Time 

the  Time 

0 

0 

0 

6 

4 

Comments : 

d.  If  you  participated  in  follow-ups  of  the  DSARC  III 
milestone,  did  these  follow-ups  generate  facts  which 
were  useful  in  improving  the  production  process? 

Please  circle  your  response. 

None  of  Some  of  Half  of  Most  of  All  of 

the  Time  the  Time  the  Time  the  Time  the  Time 
0  0  16  3 

Comments: 

e.  Do  you  feel  future  PRRs  on  foreign  contractors  should 
be  extended  beyond  the  DSARC  III  milestone? 

Yes  10  No  0 


Comments : 

2.  All  foreign  firms  reviewed  in  the  foreign  PRR  program 
were  subcontractors  to  some  U.S.  firm.  Furthermore, 
General  Dynamics  and  Pratt  6  Whitney  were  both  primes  to 
the  government  for  overall  F-16  coproduction. 

a.  Do  you  personally  feel  the  primes  and  associated 
U.S.  contractors  showed  more  or  less  interest  than 
the  SPO  team  in  those  production  areas  examined  by 
the  PRR? 
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Please  circle  your  response. 


A  Great 

Somewhat 

About  the 

Somewhat 

A  Great 

Deal  Less 

Less 

Seune 

More 

Deal  More 

0 

Comments : 

4 

6 

0 

0 

b.  Do  you  feel  the  Primes  could  have  accomplished  the 
seune  objectives  as  successfully  as  the  SPO  PRR 
tecuns? 


Yes  4 


No  6 


Comments : 

3.  The  ultimate  goal  of  the  PRR  process  is  to  reduce  the 
risk  to  the  government  when  entering  the  production 
phase.  The  official  objective  of  the  PRR  as  stated  in 
DOD  directive  5000.38  is  to  verify  that  production 
planning  has  progressed  to  a  point  where  the  production 
phase  may  be  entered  without  incurring  unacceptable 
risks.  The  PRR  must  accomplish  this  "fact  finding"  as  a 
minimum  requirement,  but  the  PRR  team  has  the  option  of 
going  beyond  this  point  if  they  feel  it  is  necessary. 

In  fact,  sound  management  may  call  for  an  extension  of 
the  PRR  process  beyond  the  information  gathering  stage. 

a.  When  performing  your  PRR,  do  you  feel  you  spent  the 
greater  part  of  your  time  and  energy  gathering 
Information  on  the  contractor's  production  readiness, 
or  helping  the  contractor  solve  troublesome  produc¬ 
tion  problems? 

Gathering  Info  5  Solving  Problems  5 

Comments : 

b.  While  you  were  visiting  the  foreign  contractor,  did 
you  spend  most  of  your  time  as  a  production  inspector 
(problem  identifier)  or  as  a  management  consultant 
(problem  solver)? 


Please  circle  your  response. 


Production 

Mostly 

About 

Mostly 

Management 

Inspector 

Production 

the 

Management 

Consultant 

Only 

Inspector 

Seune 

Consultant 

Only 

0 

2 

4 

4 

0 

Conunents : 
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c.  Do  you  feel  the  PRRs  in  which  you  participated  were 
welcomed  by  the  foreign  contractor,  or  imposed  upon 
him? 


Welcomed  7  Imposed  3 

Comments : 

d.  Were  the  PRRs  you  participated  in  adequately  intro¬ 
duced  and  marketed  to  the  foreign  contractor? 

Yes  9  No  1 


Comments : 

4.  Subcontracting  with  a  foreign  firm  presents  some  unique 
management  challenges.  In  some  cases,  there  may  be 
very  little  difference  between  a  foreign  and  domestic 
firm.  In  other  cases,  there  may  be  a  great  disparity 
between  foreign  and  domestic  firms.  Another  consideration 
is  the  variability  between  individual  firms  in  the 
United  States  may  be  greater  than  the  variability  between 
foreign  and  domestic  firms. 

When  dealing  with  a  foreign  contractor,  do  you  feel 
there  are  certain  areas  that  are  significantly  more 
important  than  when  dealing  with  a  domestic  contractor? 

Do  these  areas  require  additional  emphasis  during  a 
foreign  PRR?  If  you  feel  such  areas  exist,  please 
identify  the  three  areas  in  which  the  greatest  difference 
exists.  Comment  on  each. 

5.  During  the  PRRs  in  which  you  participated,  was  there  an 
experience  or  finding  that  illustrates  the  value  of  the 
PRR  process?  Was  there  an  event  that  generated  facts, 
information,  or  improvements  which  benefited  the  overall 
production  readiness  of  the  contractor? 

Please  briefly  relate  your  most  memorable  or  most  valu- 
£tble  PRR  experience. 
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1.  Milestones  which  demonstrate  the  achievement  of  a  prac¬ 
tical  and  producible  engineering  design  have  been  met. 

2.  Engineering  problems  encountered  during  development  have 
been  resolved  with  appropriate  tradeoffs  against  stated 
operating  requirements  so  that  production  costs/schedules 
are  optimized. 

3.  Critical  production  and  engineering  and  production  tool¬ 
ing  have  been  demonstrated  to  provide  that  engineering 
has  been  satisfactorily  accomplished. 

4.  Acquisition  will  smoothly  transition  from  full-scale 
development  to  production. 

5.  System  configuration  has  been  reviewed  to  determine  if 
any  significant  design  changes  will  be  required  for 
manufacturing . 

6.  Adequate  advance  production  planning  has  been  acconplished 
and  required  production  controls  established  to  ensure 
timely  production. 

7.  A  systematic  approach  to  standardization  has  been  accom¬ 
plished  in  the  design  process  and  parts  selection  to  . 
maximize  the  use  of  military  standard  components/  parts/ 
and  processes  consistent  with  the  system  requirements. 

8.  Product  assurance  controls  and  tests  to  prevent  manu¬ 
facturing  degradation  of  performance  parameters  have 
been  established. 

9.  Assurance  of  the  readiness  of  the  manufacturing  and  pro¬ 
duction  equipment,  methods,  facilities,  test  and  train¬ 
ing  equipment,  and  status  of  accessory  and  ancillary 
items. 

10.  Planned  production  schedules  reflect  economy  of  opera¬ 
tions  and  minimize  financial  commitments  until  all 
major  problems  have  been  resolved. 

11.  A  thorough  assessment  of  the  make-or-buy  structure  has 
been  accon^lished  and  procedures  exist  so  control  and 
visibility  of  the  vendors  and  subcontractors  can  be 
effectively  managed. 

12.  Change  activity  during  development  has  been  evaluated  and 
the  impact  of  outstanding  changes  on  production  has  been 
assessed. 
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13.  Results  of  technical  reviews  and  the  production  impact 
of  unresolved  problems  and  risks  have  been  assessed. 


14.  Test  program  results  and  the  status  of  qualification  to 
determine  production  risks  and  risk  have  been  evaluated. 

15.  Specifications  and  drawings  have  been  reviewed  to  assure 
their  adequacy  for  the  planned  production  phase. 

16.  Application  of  production  tooling  and  test  equipment  to 
manufacturing  during  development  has  been  assessed  and 
the  application  of  same  to  the  production  phase  has 
been  defined. 

17.  Material  management  system  for  determination  of  require¬ 
ments,  procurement,  receiving,  inspection,  materials 
handling  and  storage,  inventory  control,  control  of 
finished  goods,  and  shipment  is  adequate. 

18.  Production  management  systems  used  for  providing  manage¬ 
ment  with  timely  production  status  information  are 
effective. 

19.  Production  or  manufacturing  capabilities  of  major  sub¬ 
contractors  and  vendors  have  been  technically  evaluated 
and  found  adequate. 

20.  Constraints  of  laboratory  or  model  shop  capabilities 
versus  quantity  production  requirements  have  been  fully 
considered. 

21.  Quality  controls  and  inspection  procedures  have  been 
established  for  materials  treatment  or  processes  to  be 
used  in  production. 

22.  Assessment  of  the  GFP  or  services  requirements,  controls, 
management,  and  availability  of  suppliers  has  been 
accomplished. 

23.  Availability  of  production  labor  skill  requirements  has 
been  assessed  emd  their  acquisition  adequately  planned. 

24.  The  contractor  is  adequately  organized  to  accomplish  the 
production  requirements. 

25.  Planning  has  been  made  to  assure  timely  release  of 
manufacturing  instructions. 
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APPENDIX  C 

AGGREGATE  PRIORITY  LIST  OF  THE  TWENTY-FIVE 
AFSCR  84-2  PRR  QUESTIONS 
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I 


Barnett 

& 

Wales 

Ranking 

1 


2 


3 


4 


5 


6 


7 


Brechtel 

& 

Lathrop 

Ranking 


AFSCR  84-2  PRR  Questions 
With  Original  Number 


8  9.  Assurance  of  readiness  of  the 

manufacturing  and  production 
equipment,  methods,  facilities, 
test  and  training  equipment,  and 
status  of  accessory  and  ancillary 
items. 


1 


18 


6.  Adequate  advanced  production 
planning  has  been  accomplished 
and  required  production  controls 
established  to  ensure  timely 
production . 

18.  Production  management  systems 

used  for  providing  management  with 
timely  production  status  informa¬ 
tion  are  effective. 


24 


23 


7 


19 


23.  Availability  of  production  labor 
skill  requirements  has  been 
assessed  and  their  acquisition 
adequately  planned. 

24.  The  contractor  is  adequately 
orgemized  to  accomplish  the  pro¬ 
duction  requirements. 

19.  Production  or  manufacturing 

capedsilities  of  major  subcontrac¬ 
tors  and  vendors  have  been  tech¬ 
nically  evaluated  and  found 
adequate . 

17.  Material  management  system  for 
determination  of  requirements, 
procurement,  receiving,  inspec¬ 
tion,  materials  handling  and 
storage,  inventory  control,  con¬ 
trol  of  finished  goods,  and 
shipment  is  adequate. 
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Barnett 

& 

Wales 
Rankin 


Brechtel 
&  : 

Lathrop 
Ramkin 


AFSCR  84-2  PRR  Questions 
With  Original  Number 

Critical  production  engineering 
euid  production  tooling  have  been 
demonstrated  to  prove  that  engi¬ 
neering  has  been  satisfactorily 
accomplished. 

Application  of  production  tooling 
and  test  equipment  to  manufac¬ 
turing  during  development  has 
been  assessed  and  the  application 
of  Scune  to  the  production  phase 
has  been  defined. 

Planning  has  been  made  to  assure 
timely  release  of  manufacturing 
instructions. 

Change  activity  during  development 
has  been  evaluated  and  the  impact 
of  outstanding  changes  on  pro¬ 
duction  has  been  assessed. 

Quality  controls  and  inspection 
procedures  have  been  estedilished 
for  materials  treatment  or  pro¬ 
cesses  to  be  used  in  production. 

Milestones  which  demonstrate  the 
achievement  of  a  practical  and 
producible  engineering  design 
have  been  met. 

Assessment  of  the  GFP  or  service, 
requirements ,  controls ,  manage¬ 
ment,  and  availability  of  suppliers 
has  been  accomplished. 

Results  of  technical  reviews  and 
the  production  impact  of  unre¬ 
solved  problems  and  risk  have 
been  assessed. 

System  configuration  has  been 
reviewed  to  determine  if  any  sig¬ 
nificant  design  changes  will  be 
required  for  manufacturing. 


[l. 


Barnett 

r 

Brechtel 

r. 

a 

Wales 

Reuiking 

Qt 

Lathrop 

Ranking 

AFSCR 

With 

84-2  PRR  Questions 
Original  Number 

Specifications  and  drawings  have 
been  reviewed  to  assure  their 
adequacy  for  the  planned  pro¬ 
duction  phase. 

Product  assurance  controls  and 
tests  to  prevent  manufacturing 
degradation  of  performance  para¬ 
meters  have  been  established. 

Test  program  results  and  the 
status  of  qualification  testing 
to  determine  production  impact 
and  risk  have  been  evaluated. 

A  thorough  assessment  of  the 
make-or-buy  structure  has  been 
accomplished  and  procedures  exist 
so  control  and  visibility  of  the 
vendors  and  subcontractors  can  be 
effectively  maintained. 

Acquisition  will  smoothly  transi¬ 
tion  from  full-scale  development 
to  production. 

Engineering  problems  encountered 
during  development  have  been 
resolved  with  appropriate  trade¬ 
offs  against  stated  operating 
requirements  so  that  production 
costs/schedules  are  optimized. 

Planned  production  schedules 
reflect  economy  of  operations  and 
minimize  financial  commitments 
until  all  major  development  prob¬ 
lems  have  been  resolved. 

Constraints  of  laboratory  or  model 
shop  capabilities  versus  quantity 
production  requirements  have  been 
fully  considered. 


6 


25  20  7.  A  systematic  approach  to  stan¬ 

dardization  has  been  accomplished 
in  the  design  process  and  parts 
selection  to  maximize  the  use  of 
military  standard  components, 
parts,  and  processes  consistent 
with  the  system  requirements. 
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APPENDIX  D 

COMPARISON  OF  THE  RANKINGS  OF  THE  TWENTY-FIVE 
AFSCR  84-2  PRR  QUESTIONS  FROM  THE  TWO  STUDIES 
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Original 

Ranking 

Brechtel 

& 

Lathrop 

(X) 

Barnett 

& 

Wales 

(Y) 

‘^i 

(X-Y) 

d.2 

1 

1 

3 

13 

-10 

100 

2 

2 

22 

-20 

400 

3 

9 

8 

1 

1 

4 

10 

21 

-11 

121 

5 

5 

16 

-11 

121 

6 

1 

2 

-1 

1 

7 

20 

25 

-5 

25 

8 

25 

18 

7 

49 

9 

8 

1 

7 

49 

10 

11 

23 

-12 

144 

11 

14 

20 

-6 

36 

12 

12 

11 

1 

1 

13 

4 

15 

-11 

121 

14 

6 

19 

-13 

169 

15 

13 

17 

-4 

16 

16 

17 

9 

8 

64 

17 

19 

7 

12 

144 

18 

18 

3 

15 

225 

19 

7 

6 

1 

1 

20 

21 

24 

-3 

9 

21 

15 

12 

3 

9 

22 

22 

14 

8 

64 

23 

24 

4 

20 

400 

24 

23 

5 

18 

324 

25 

16 

10 

6 

36 

2630  TOTAL 
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APPENDIX  E 

F-16  SPO  EXPERT  RANKINGS  FOR  THE  TWENTY-FIVE 
STANDARD  AFSCR  84-2  PRR  QUESTIOMSl 
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TWENTY-FIVE  AFSCR  84-2  PRR  QUESTIONS 


Format  for  this  Appendix  taken  from  the  unpublished  master's  thesis  of 
Brechtel  and  Lathrop  (4:134-135). 


APPENDIX  F 


FOREIGN  PRR  EXPERTS 


Capt  Donald  L.  Brechtel 
2717  Bedford  Way 
Tallahassee  FL  32308 


Major  Leo  Pavlow 
AFCMD/EPP 

Kirtland  AFB  NM  77117 


Mr  Burton  W.  Clemens 
159  Mira  Velero 
San  Clemente  CA  92692 


Lt  Col  Lavelle  Prine 
Commander,  TUSLOG  Det  93 
P.O.  Box  1 
APO  New  York  09051 


Lt  Col  Thomas  Fiorino  Mr  Albert  F.  Schub 

ASD/yPM  3221  Veteran  Avenue 

Wright-Patterson  AFB  OH  45433  Los  Angeles  CA  90034 


Mr  Jack  Fremks 
31496  W.  Nine  Drive 
Laguna  Niguel  CA  92677 


Mr  Wayne  Stafford 
2033  Baltra  Place 
Costa  Mesa  CA  92626 


Major  Pete  Giusti 
AFSC/PMDP 

Andrews  AFB  DC  20334 


Mr  Ernest  Vitaliani 
ASD/YPMA 

Wright-Patterson  AFB  OH  45433 
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APPENDIX  G 

THE  KENDALL  RANK  CORRELATION  TAU  PROCEDURE 
AND  CALCULATIONS 
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1.  The  list  of  X  and  Y  rankings  are  shown  with  the  X  ranking 
arranged  in  natural  order.  The  X  rankings  are  the  results 
of  the  meem  rankings  of  the  twenty- five  PRR  questions  by 
Brechtel  and  Lathrop  (4:144-147),  and  the  corresponding  Y 
rankings  are  the  results  of  the  mean  rankings  of  the  twenty- 
five  PRR  questions  by  this  work  (Appendix  E) . 


X 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


_y 

2 

22 

13 

15 

16 

19 
6 
1 
8 

21 

23 
11 
17 

20 
21 
10 

9 

3 

7 

25 

24 


1X6 


X  Y 

22  14 

23  5 

24  4 

25  18 

2.  S  is  determined  by  starting  with  the  first  number  at  the 
top  of  the  Y  ranks  and  counting  the  number  of  rankings  to  the 
bottom  which  are  larger.  Then  subtract  from  this  the  number 
of  rankings  which  are  smaller  than  the  starting  number.  Do 
this  for  all  ranks  by  going  sequential  down  the  list  and 
then  summing  the  results.  The  computation  is  shown  below: 

S  =  (23-1)  +  (3-20)  +  (11-11)  +  (9-12)  +  8-12)  +  (5-14)  + 
(14-4)  +  (1-17)  +  (12-4)  +  (3-12)  +  (2-12)  +  (7-6)  + 

(4-8)  +  (2-9)  +  (4-6)  +  (4-5)  +  (4-4)  +  (7-0)  +  (4-2)  + 
(0-5)  +  (0-4)  +  (1-2)  +  (1-1)  +  (1-0)  =  -42 

3.  Ns  number  of  questions  ranked  on  both  X  or  Y. 

N  =  25 

4.  The  following  formula  was  used  to  calculate  the 
Kendall  Tau: 


S 

"  i  N  (N-1) 
=  -0.14 
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APPENDIX  H 

SELECTED  RESPONDENT  COMMENTS  ON 
SELECTED  QUESTIONS 
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This  appendix  provides  additional  comments  on  selected 
questions  which  were  included  on  the  completed  question- 
naires.  These  solicited  comments  provide  additional  insight 
into  the  perceptions  of  the  foreign  PRR  experts.  Alterations 
of  the  comments  were  not  made  with  the  exception  of  deleting 
a  respondent's  name. 

1.  The  first  group  of  comments  pertain  to  question 
la  in  the  questionnaire  which  determined  the  respondent's 
feelings  regarding  whether  the  PRR  process  should  terminate 
at  the  DSARC  III  milestone  or  continue.  All  ten  respondents 
felt  the  PRR  process  did  not  terminate  at  the  DSARC  III  mile¬ 
stone. 

COMMENTS 

"We  had  a  number  of  follow-up  areas  and  open  action 
items  to  review  (after  DSARC  III).” 

"Follow-ups  have  continued  throughout  the  programs 
and  have  been  essential  to  good  management." 

"New  starts,  vendors,  etc.,  require  careful  review 
after  DSARC  III.  Many  PRR's  require  follow-up  for  some  time 
after  DSARC  III  and  well  into  the  first  production." 

2.  The  next  comment  pertains  to  question  Ic  which 
surveyed  the  respondents'  feelings  concerning  the  aUsility  of 
the  PRR  to  generate  facts  that  were  useful  during  the  pro¬ 
duction  readiness  decision.  The  ten  respondents  answered 


that  either  most  of  the  time  or  all  of  the  time  the  PRR 
generated  useful  facts. 


COMMENT 


"The  readiness  decision  was  based  to  a  great  extent 
on  the  2.5  hour  briefing  which  displayed  the  information 
from  PRR  visits." 


3.  The  next  comment  pertains  to  question  Id  which 
surveyed  the  respondents'  feelings  regarding  the  value  of 
follow-up  visits  to  the  contractors  after  DSARC  III. 


COMMENT 


"Follow-ups  kept  pressure  on  U.S.  contractors  and 
foreign  contractors  to  more  efficiently  communicate  and  to 
operate  more  in  a  partnership  role  vs.  a  normal  subcontractor 
role.  The  visibility  of  a  coproduction  program  is  viewed  at 
very  high  levels  of  host  governments,  as  such,  requires 
dedicated  efforts  to  identify  work  problems  from  the  onset." 


4.  The  following  comments  pertain  to  question  le 
which  surveyed  the  respondents'  feelings  concerning  the 
extension  of  future  PRRs  beyond  DSARC  III.  All  ten  experts 
felt  PRRs  should  continue  past  DSARC  III. 


COMMENTS 


"The  amount  of  surveillance  needed  is  not  an  easy 
thing  to  measure. . .but  it  will  always  be  necessary  to  conduct 
PRs  whenever  a  schedule  slippage  appears.  I  realize  this  is 
not  a  PRR,  but  it  is  closely  related  and  the  function  should 
be  continued." 

"As  Appropriate  to  assess  production  readiness  in  the 
cases  of  major  progr£un  changes  (e.g.,  increases  in  workload, 
FMS  sales,  major  design  changes,  etc.)." 
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"A  modified  PRR  limited  to  producibility ,  for 
example,  is  needed  beyond  the  milestone  to  surface  any  new 
or  potential  problems  not  apparent  in  pre-production  or 
early  production  deliveries." 

"But  only  where  a  discrepancy  or  action  item  initi¬ 
ated  before  DSARC  III  needs  a  follow-up  for  closure.  PRR 
needs  to  give  way  to  normal  surveillance." 

"Continuity  of  PRR  and  follow-up  would  be  assured 
if  both  were  attended  by  the  same  person." 


5.  The  following  comments  pertain  to  question  2a 
which  surveyed  the  respondents '  feelings  regarding  whether 
the  primes  and  associated  U.S.  contractors  showed  more  or 
less  interest  than  the  SPO  teams  in  the  production  areas 
examined  by  the  PRR.  The  responses  indicated  that  the 
respondents  felt  the  Primes  and  associated  U.S.  contractors 
showed  either  somewhat  less  interest  or  about  the  same 
amount  of  interest. 


COMMENTS 


"Some  contractors  were  not  candid  in  their  help  to 
the  foreign  producer.  They  did  not  exchange  all  their 
information  they  had  available  to  assist  the  contractor  in 
solving  any  problems  (direct/indirect)." 

"It  does  appear  to  be  a  function  of  the  dependency 
of  the  prime  on  his  sub  for  maintenance  of  schedule  deliv¬ 
eries  as  to  the  degree  of  interest." 


6.  The  following  comments  pertain  to  question  2b 
which  surveyed  the  respondents'  feelings  concerning  the 
capability  of  the  Primes  to  accomplish  the  same  objectives 
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as  those  of  the  SPO  teams  in  a  PRR.  The  majority  of  the 
respondents  believed  the  Prime  could  not  accomplish  the  same 
objectives  as  the  SPO  teaims. 


COMMENTS 


"SPO  teams  provide  a  different  viewpoint  than  the 
Primes  amd  serve  as  a  "customer’s"  attitude  instead  of  a 
"partner"." 

"The  Primes  had  a  marketing  incentive  to  make  it 
successful,  as  such  they  did  a  good  job  of  identifying  prob¬ 
lems  and  worked  with  the  foreign  company  to  resolve  them." 

"The  PRRs  were  done  to  (1)  aid  in  the  government 
decision  to  proceed,  and  (2)  correct  deficient  areas  of 
readiness.  Primes  could  not  accomplish  (1)  but  could  do  as 
well  in  (2)  if  motivated." 

"The  F-16  SPO  personnel  (as  U.S.  government  repre¬ 
sentatives)  as  a  whole  probcibly  attained  the  PRR  objectives 
more  efficiently  and  effectively  because  of  the  political 
nature  of  the  F-16  program." 

"Yes  they  could  have... but  I  don't  believe  the 
examination  would  have  been  as  close  or  as  exacting.  The 
"outside"  questions  were  stimulating  in  many  cases  and 
caused  some  better  procedures  and  methods  to  be  adopted." 


7.  The  following  comments  pertain  to  question  3a 
which  surveyed  the  respondents'  feelings  on  whether  they 
spent  more  time  and  energy  gathering  facts  or  more  on  solving 
problems.  The  respondents  were  split  50/50  on  the  question. 


COMMENTS 


”We  identified  the  problem  areas.  For  the  most  part, 
we  worked  with  Primes  to  solve  the  problems.  Sometimes  we 
gave  direct  advice." 
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"Since  management  systems  in  planning  and  controlling 
work  was  one  of  the  weak  areas,  it  required  much  effort  in 
explaining  needs  to  improve  these  areas . " 

"The  information  was  generally  available  but  the 
solutions  had  to  (be)  worked  out  in  detail  with  the  govern- 
ment  serving  as  technical  consultants." 

" I  don ' t  know  how  a  PRR  team  can  gather  information 
on  the  contractor's  readiness  without  finding  problems  in 
his  systems.  Therefore,  the  answer  is  yes  to  both  questions." 

"I  feel  the  question  3  Part  II  cannot  be  answered 
without  explanation: 

First,  in  very  few  cases  have  we  been  problem  solvers 
for  the  contractors.  We  have  and  do  make  suggestions  some¬ 
times  referring  them  to  others  we  may  know  of  that  have  had 
similar  problems.  There  have  been  times  our  suggestions  on 
basic  manufacturing  and  parts  supply  systems  have  helped  to 
mitigate  chronic  production  delays.  Our  greatest  function 
on  the  PRRs  and  manufacturing  program  follow-up  has  been 
that  of  evaluation.  Obviously  there  needs  to  be  much 
gathering  of  information  to  make  a  sound  evaluation. 

Our  summary  reports  reflect  our  assessment  of: 

Whether  sufficient  and  adequate  facilities  are  in  place. 

Whether  those  to  be  acquired  have  sufficient  lead  time  not 
only  to  procure  but  to  debug  and  to  train  operators. 

Whether  production  control  systems  in  use  are,  in  our 
opinion,  adequate  for  the  task. 

Whe^er  or  not  area  has  sufficient  labor  available,  what 
training  will  be  required,  what  are  the  unique  requirements 
from  the  governments  in  respect  to  hiring,  firing,  laying 
off,  working  overtime  and  second  and  third  shifts. 

Whether  the  planned  for  lead  times  are  overly  optimistic  or 
too  loose. 

Whether  a  schedule  slippage  has  a  real  chance  of  being 
recovered  in  euiy  kind  of  a  reasonable  time  (i.e.,  restric¬ 
tions  on  O.T.  emd  shift  work) . 

Whether  second  and  third  shift  operations  may  be  instituted 
to  reduce  elapsed  times  for  limited  periods. 
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what  prior  experience  has  the  management  had  in  working  as 
a  sub  contractor,  in  monitoring  and  reporting  to  a  prime. 

Will  he  need  to  be  especially  monitored  to  see  that  he 
understands  the  need  for  anticipatory  reporting  to  alert 
others  outside  of  his  facility  of  potential  short  falls  so 
that  alternate  planning  may  be  made. 

Will  program  management  concepts  be  an  annoyance  to  the 
supplier  or  will  he  readily  accept  and  develop  a  viable 
program  control  methodology. 

What  is  the  work  pace  you  see  in  the  factory? 

How  is  the  housekeeping? 

How  are  the  parts  handled  and  protected? 

What  controls,  inspection  systems  are  in  evidence  to  maintain 
quality. 

Whether  there  is  evidence  that  real  planning  for  production 
has  and  is  being  accomplished  or  whether  the  planning  briefed 
to  the  teams  was  just  an  exercise  to  satisfy  the  requirement. 

When  giving  recovery  forecasts,  what  chances  are  there  for 
making  recovery  commitments.  What  is  the  demonstrated 
history. 

One  could  continue  for  some  time  citing  examples,  but  it  is 
from  this  type  of  data  obtained  verbally,  visually  and  from 
records  that  evaluations  are  made.  What  one  looks  at  and  in 
what  depth  depends  on  the  program,  product,  complexity,  and 
suppliers  past  experience  and  what  you  see  and  hear  when 
you  are  on  his  premises. 

I  believe  our  main  contributions  to  the  F-IS  SPO  has  been 
that  of  evaluating  the  data  fed  us  by  the  supplier,  that  which 
we  collected  and  from  those  observations  we  have  made  when 
visiting  the  manufacturing  areas.  Such  evaluations  have 
covered  everything  from  production  readiness  to  schedule 
recovery  prospects,  from  adequacies  of  tooling  to  selection 
of  fedsrication  shop  equipment,  from  manufacturing  management/ 
control  production  management  concepts  from  production 
testing  to  equipment  utilization,  etc. 

Thus  the  answers  to  Part  ZI  question  3  when  circling  the 
response  do  not  truly  reflect  the  activity  which  is  a  little 
"problem  identifying"  and  some  "problem  solving"  and  mostly 
the  evaluation  needed  for  the  program  office  manufacturing 
directorate  to  manage  the  manufacturing  progreun  in  as  crises 
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free  manner  as  possible,  anticipating  problem  areas  and 
planning  accordingly  both  in  terms  of  actions  and  follow-up. 

I  realize  that  the  questionnaire  is  directed  at  PRRs  and  my 
comments  have  been  expanded  to  cover  MMPR's,  PRR's,  MPR's 
and  post  DSARC  III  activity  but  the  elements  discussed  can 
be  limited  to  PRR's  and  still  be  valid." 


8.  The  following  comments  pertain  to  question  3c 
which  surveyed  the  respondents'  feelings  regarding  whether  or 
not  the  foreign  contractors  welcomed  the  PRR  visit.  The 
majority  of  the  responses  suggested  that  the  foreign  con¬ 
tractor  welcomed  the  PRR  team  visit. 


COMMENTS 


"Initially  it  was  imposed  because  foreign  companies 
were  not  used  to  this  requirement.  Later  these  types  of 
reviews  were  accepted  but  not  necessarily  welcomed." 

"The  Europeans  were  worried  about  us  until  they 
found  out  that  we  were  not  trying  to  tell  them  how  to  run 
their  business  and  that  we  were  there  as  partners  to  make 
the  program  work . " 

"At  first  there  was  some  reluctance.  This  was  over¬ 
come  by  limiting  team  size,  assuring  the  competence  and  exper 
tise  of  the  government  team  was  the  best  available,  and 
recognizing  the  customs  and  attitudes  of  the  host  con¬ 
tractor  . " 


"Initially  they  felt  in^osed  upon,  but  as  we  worked 
with  them  most  of  them  became  more  receptive  when  they 
found  out  that  we  were  not  there  to  criticize  but  to  help 
them. " 


"These  were  new  to  most  contractors  and  as  such, 
somewhat  feared.  They  all  admitted  at  the  conclusion  that 
they  were  beneficial." 


125 


9.  The  following  conments  pertain  to  question  3d 
which  surveyed  the  respondents'  feelings  concerning  whether 
the  PRR  was  adequately  introduced  to  the  foreign  contractor 
or  not.  Most  respondents  felt  they  were. 

COMMENTS 


"Not  early  in  the  program,  but  since  that  time 
foreign  companies  have  seen  this  can  be  a  way  to  help 
resolve  difficult  management  issues." 

"We  recognized  early  that  protocol  was  a  significant 
factor  to  assure  the  acceptance  and  the  proper  atmosphere.” 

"The  foreign  firms  were  adequately  informed  about 
the  F-16  PRR  objectives.  As  a  whole,  the  F-16  PRR  teauns 
experienced  less  resistance  from  foreign  contractors  than 
was  experienced  from  counterpart  U.S.  contractors  during 
some  F-16  PRRs  conducted  in  the  U.S." 


10.  The  following  comments  pertain  to  question  4 
which  surveyed  the  respondents'  opinion  relating  to  what 
areas  were  felt  to  be  the  most  important  in  dealing  with  a 
foreign  contractor  as  opposed  to  the  domestic  contractor. 


COMMENTS 


"An  understanding  of  the  social  legislation  imposed 
as  a  means  of  assuring  maximum  employment,  i.e.,  laws 
relating  to  dismissing  employees,  limitations  on  amount  of 
overtime  that  may  be  worked  and  the  financial  impact  on  the 
employees  whose  social  benefits  can  be  reduced  by  increased 
overtime  income  resulting  in  a  negative  condition." 

"A  fundamental  cultural  difference  in  attitude  towards 
job,  i.  e. ,  enjoyment  of  life  is  paramount  and  job  is  a  means 
to  this  end.  Thus  schedules  are  goals  and  not  sacra-sanct. 
Night  shifts  are  to  be  avoided  and  all  holidays  will  be 
honored  and  observed." 
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"Usually  bluntness  and  overly  direct  approaches  -are 
resented.  A  more  formal  and  diplomatic  approach  following 
the  prevailing  social  customs  often  yields  more,  both  in 
respect  and  in  responses." 

"The  greatest  impact  is  on  the  elapsed  times  required 
to  complete  tasks.  One  must  be  aware  of  this  fact  and  con¬ 
sider  it  in  all  plans,  schedules  and  delinquencies." 

"Middle  management  in  foreign  firms  is  usually  non¬ 
existent  or  at  best  very  lean.  Therefore,  the  manager  has  a 
closer  vantage  of  production  problems  and  requires  greater 
technical  skills  to  solve  a  problem  since  a  large  advisory 
staff  does  not  exist." 

"Usually,  machinery  amd  tooling  procured  in  Europe 
are  required  to  make  a  test  piece  to  print  before  acceptance 
at  builder  locations.  This  process  allows  for  debugging 
before  shipment.  U.S.  firms  can  take  a  lesson  from  this 
experience . " 

"LcUsor  is  a  much  more  complicated  problem  than  in 
U.S. A.  It  is  very  political  and  demanding  to  company  manage¬ 
ment  and  is  a  factor  more  con^jlicated  than  we  encounter. 

Their  production  plans  are  impacted  by  this  and  must  be 
analyzed  differently  from  U.S. A.  PRR's." 

"Internal  laws  of  distributing  work  vary  in  each 
country.  Here  again  their  PRR  will  read  differently  to  a 
U.S. A.  one." 

"Memy  exotic  materials  are  not  available  internally 
or  even  in  Europe,  therefore,  procurement  plans  for  parts 
and  material  have  to  be  minutely  analyzed." 

"Labor  unions  and  foreign  government  influences  on 
foreign  contractors'  management  prerogative  (e.g.,  single 
shift  operation  is  standard  operating  procedure  for  foreign 
firms  in  Europe) . " 

"Sources  of  supply  for  material,  parts,  etc.  (i.e., 
decision  between  U.S.  sources  or  foreign  sources  of  supply) . 
Action  laws  and  transportation  plans  { for  parts ,  etc . ) 
among  foreign  firms  and  with  U.S.  contractor  counterparts. 
Drawing  conversion  activity  (i.e.,  to  metric).” 

"Front  End  Coordination  and  Planning  -  Visits  should 
be  justified  and  planned  well  in  advance  wi^  care.  Coordi¬ 
nation  with  all  concerned  parties  is  mandatory.  The  cogni¬ 
zant  government  people  in  the  host  country  including  the 
Armed  Forces,  the  U.S.  Embassy,  Resident  U.S.  people,  the 
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Prime  Contractor,  the  U.S.  Coproducer.  Limit  the  size  of 
the  visiting  group  strictly.  Review  the  agenda  carefully 
for  necessity  of  items." 

"Dealing  with  Foreign  Contractor's  Management  -  You 
must  be  accepted  by  the  general  manager.  They  have  few 
middle  managers.  Be  prepared  to  deal  with  the  G.M.  the 
whole  time.  Select  an  agenda  appropriate  to  his  attention. 
Allow  the  G.M.  to  select  topics  and  areas  to  review.  Do  not 
force  the  conduct  of  visit  to  your  desires  while  there.  All 
that  should  be  precoordinated." 

"Don't  Play  the  Ugly  American  -  Their  systems  and 
procedures  are  time-tested.  Don ' t  underestimate  their 
ability.  Relocate  their  approach  and  take  the  time  to  see 
results.  Recognize  their  labor  practices  and  the  peculiar 
problems  they  present.  Don't  try  to  make  them  change  just  to 
be  like  U.S.  firms." 

"Unions  in  Europe  are  very  strong.  They  only  work 
one  shift  and  5  days  per  week  consequently." 

"Memufacturing  spare  times  are  up  to  12-14  months 
longer  than  the  U.S.  which  causes  lead  time  problems  and 
control  problems." 

"Europeans  are  not  used  to  our  year-to-year  funding 
and  would  like  to  be  put  on  contract  for  the  full  contract 
number  of  airplanes  even  though  this  spans  many  years.  They 
are  reluctant  to  spend  money  on  facilities  without  this 
assurance. " 

"The  domestic  contractors  must  take  more  care  in 
reviewing  the  basis  for  the  foreign  contractor  planning.  In 
meuiy  cases  the  foreign  contractor  planning  is  based  on 
invalid  assumptions  due  to  lack  of  understanding  or  use  of 
U.S.  contractor  standards  which  may  not  be  applicable  to  the 
foreign  contractors . " 
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APPENDIX  I 


SELECTED  UNSOLICITED  RESPONDENT  COMMENTS 


This  appendix  provides  additional  unsolicited  comments 
that  were  received  with  the  questionnaire  answer  sheets 
returned  by  the  respondent.  The  inclusion  of  a  sampling  of 
the  candid  comments  in  this  appendix  provides  additional 
insight  into  the  perceptions  of  the  foreign  PRR  experts. 
Alterations  of  the  comments  were  not  made  with  the  exception 
of  deleting  the  respondents'  naunes. 


Enclosed  Comments 


"COMMENTS  ON  PRR  FUNCTION 
Preplanning  a  PRR  . 

(1)  A  specific  series  of  subjects/questions  should  be  gen¬ 
erated  and  agreed  upon  between  the  SPO  and  the  firm  to  be 
visited. 

(2)  The  number  of  items  should  be  kept  at  a  minimum  and  must 
be  related  only  to  the  situations  under  examination.  For 
example:  Some  firms  will  dwell  on  how  great  they  were  in 
the  past  on  certain  programs.  This  type  of  information 
(salesmemship?)  is  not  needed  by  the  PRR  teeun  and  is  a  waste 
of  time  for  all  concerned. 

(3)  After  all  presentations  are  complete  and  questions  by  the 
PRR  team  have  been  answered,  the  SPO  teeun  should  evaluate  all 
the  information  (privately)  and  then  present  the  firm  with 
their  conclusions  or  recommendations.  This  is  the  outbrief- 
ing. 

(4)  A  date  should  then  be  estedslished  for  follow-up  action 
if  needed." 

"Most  of  the  foreign  firms  that  were  PRR'd  conduct 
their  business  in  basically  the  same  manner.  However,  since 
many  of  them  were  in  a  similar  (to  them)  Line  of  business 
than  was  imposed  by  the  F-16,  they  found  themselves  having 
to  make  many  changes  in  methods  and  operations.  These 
changes  were  new  to  them  and  were  not  always  sufficient  and 
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in  some  cases  were  not  always  adopted  in  time.  3ome  failed 
simply  because  they  were  not  aggressive  enough  in  meeting 
their  own  dates  to  make  the  changes. 

Many  of  the  foreign  firms  decided  the  P-16  require¬ 
ments  was  business  as  usual  and  this  attitude  had  a  somewhat 
negative  effect  on  the  preplanning  and  resultant  startup  of 
their  program.  Exeunple:  One  firm  in  Denmark  had  a  thriving 
and  successful  business  of  producing  telephone  equipment. 

It  was  assembly  line  type  of  work  and  yet  was  not  related  to 
what  was  required  for  F-16  flite  computers.  We  continually 
advised  them  during  the  PRR  of  the  complexity  they  would  be 
getting  into  and  how  they  would  have  to  change  their  pro¬ 
cedures  to  build  the  components  for  the  F-16  at  the  scheduled 
rate. 


The  internal  systems  in  some  firms  did  not  provide 
enough  visibility  of  the  hardware  requirements  which  put  them 
in  a  delinquent  condition  from  day  one.  They  then  turned 
to  GO  for  help  which  necessitated  a  great  deal  of  support 
activity  by  coproducers  in  the  US  and  related  suppliers. 

A  number  of  firms  did  not  put  their  "best"  people  on 
the  F-16  program  and  in  some  cases  used  new  people  who  did 
not  have  the  experience  to  do  the  job.  It  also  caused  a 
condition  to  exist  where  top  management  did  not  support  the 
F-16  and  left  it  to  the  appointee  until  trouble  started  and 
then  they  used  better  management  people. 

SUMMARY 


These  are  a  few  of  the  items  that  a  PRR  team  must 
look  at  and  continue  to  review; 

(1)  Attitude  of  the  firm. 

(2)  Close  ex^unination  of  their  system,  operations, 
controls,  particularly  their  parts  system. 

(3)  Management  types  of  people.  Are  they  flexible? 
Aggressive?  Dedicated?" 

"When  doing  a  PRR  on  foreign  companies  where  the 
engineering  responsibility  is  with  the  U.S.  contractors,  the 
emphasis  must  be  placed  on: 

-  completeness  of  engineering  data  received 

-  plemning  (not  just  milestone  charts) 

-  management  systems  to  handle 

-  changes 

-  in-process  control 

-  material  control 

-  purchasing 
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-  levels  of  management  and  their  responsive¬ 
ness  to  anomalies 

-  tooling  capacity  (most  have  to  sub  for  this 
capability) .  This  is  not  a  general  comment 
for  all  European  firms  only  or  experienced 
with  the  F-16. 

-  production  control 

-  communication  channels  due  to  distance  and  mis¬ 
understandings.  " 

"The  F-16  PRR  process  was  an  excellent  opportunity 
to  bring  together  the  U.S.  government  representatives  (e.g. , 
F-16  SPO,  F-16  CASEUR,  AFPRO  personnel) ,  the  prime  contrac¬ 
tors  (GD  and  Pratt-Whitney) ,  the  U.S.  subcontractors,  and 
the  European  co-producers  together  on  the  site  where  pro¬ 
duction  activity  would  occur  to  mutually  identify  problems 
and  ascertain  corrective  actions. 

"In  addition,  the  F-16  PRR  forum  allowed  for  the 
cross-fertilization  of  ideas  eunong  U.S.  and  foreign  firms  in 
both  directions.  The  communication  process  among  all  F-16 
program  participants  was  facilitated  via  the  F-16  PRR  pro¬ 
cess.  " 


"There  are  other  criteria  that  should  be  added  to 
this  list  for  application  to  foreign  contractors  (e.g., 

F-16  co-producers) . 

For  example; 

1.  A  management  system  or  network  has  been  esteds- 
lished  to  ensure  timely  information  dissemination  eunong 
foreign  contractors  and  U.S.  contractors. 

2.  Transportation  guidelines  have  been  established 
to  ensure  the  timely  transfer  of  parts,  equipment,  etc., 
eunong  foreign  contractors  and  U.S.  contractors. 

3.  Customs  guidelines  for  shipping  among  countries 
have  been  fully  developed  to  prevent  program  deals  because 
of  inadequate  rules  or  regulations. 

4.  A  training  program  has  been  established  to  ensure 
accepteUDle  transfer  of  technology,  skills,  procedures,  etc., 
eunong  foreign  contractors  and  U.S.  contractors." 
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